From 6cf37cd1330f598c86a04252f0dd98aab42e9ca6 Mon Sep 17 00:00:00 2001 From: Jia Rong Wu Date: Fri, 7 Feb 2025 02:45:20 -0500 Subject: [PATCH 1/8] Updating with proper documentation for plugin manifests. (#5936) --- .../plugins/how-to-serve-plugins/plugin-bundles.md | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/tyk-docs/content/plugins/how-to-serve-plugins/plugin-bundles.md b/tyk-docs/content/plugins/how-to-serve-plugins/plugin-bundles.md index ffabdc4cbe..bf8edcf41c 100755 --- a/tyk-docs/content/plugins/how-to-serve-plugins/plugin-bundles.md +++ b/tyk-docs/content/plugins/how-to-serve-plugins/plugin-bundles.md @@ -55,6 +55,9 @@ A plugin bundle must include a manifest file (called `manifest.json`). The manif A sample manifest file looks like this: +- The `name` is the function to be invoked upon a specific plugin hook. +- The `path` is the name of the file which contains the function. Note that you can not have nested paths or subdirectories in the path specification. + ```json { "file_list": [ @@ -64,12 +67,14 @@ A sample manifest file looks like this: "custom_middleware": { "pre": [ { - "name": "PreMiddleware" + "name": "PreMiddleware", + "path": "middleware.py" } ], "post": [ { - "name": "PostMiddleware" + "name": "PostMiddleware", + "path": "middleware.py" } ], "driver": "python" From c6723f34e356b6f3f931c9bab0b9543a065f0771 Mon Sep 17 00:00:00 2001 From: JoanCamosTyk <157505463+JoanCamosTyk@users.noreply.github.com> Date: Mon, 10 Feb 2025 12:38:02 +0100 Subject: [PATCH 2/8] Cloud Release Notes 1.25.0.md (#5877) --- .../developer-support/release-notes/cloud.md | 324 ++++++++++-------- 1 file changed, 173 insertions(+), 151 deletions(-) diff --git a/tyk-docs/content/developer-support/release-notes/cloud.md b/tyk-docs/content/developer-support/release-notes/cloud.md index ab1a5341e6..de48155e90 100644 --- a/tyk-docs/content/developer-support/release-notes/cloud.md +++ b/tyk-docs/content/developer-support/release-notes/cloud.md @@ -4,10 +4,82 @@ date: xx description: "Release notes documenting updates, enhancements, and changes for Tyk Cloud" tags: ["Tyk Cloud", "Release notes", "v1.23", "1.23.0", "changelog"] +--- + +## 1.25.0 Release Notes + +### Release Date 10 of February 2025 + +### Release Highlights +This Tyk Cloud update enhances Gateway version management, ensuring a more streamlined, secure, and user-friendly experience. With the new UI versioning updates, users have clearer visibility into supported versions, direct upgrade recommendations, and the ability to automate version upgrades for effortless maintenance. +These changes empower teams to stay on supported, secure, and high-performing versions. +For more details, check out the [documentation on Gateway versioning and auto-upgrades](https://tyk.io/docs/developer-support/release-notes/special-releases/) + +### Breaking Changes + +There are no breaking changes in this release. + +### Downloads +- [latest version of Mserv](https://github.com/TykTechnologies/mserv/releases/latest) + +### Deprecations + +There are no deprecations in this release. + +### Changelog {#Changelog-v1.25.0} + +#### Added + + +#### Changed + + + +#### Fixed + + --- -## 1.24 Release Notes + +## 1.24.0 Release Notes ### 1.24.2 Release Notes @@ -110,211 +182,161 @@ This feature enables trace export capabilities, providing deep insights into API For more details, check out the [documentation on setting up Telemetry export]({{< ref "tyk-cloud#enabling-telemetry-in-tyk-cloud" >}}). +### Breaking Changes -#### Breaking Changes - There are no breaking changes in this release #### Downloads - [latest version of Mserv](https://github.com/TykTechnologies/mserv/releases/latest) -#### Deprecations - +### Deprecations + There are no deprecations in this release +--- ## 1.23 Release Notes -### 1.23.0 Release Notes - -#### Release Date 14 of November 2024 +### Release Date 14 of November 2024 -#### Release Highlights +### Release Highlights This Tyk Cloud update introduces features that improve both flexibility in plugin management and user onboarding. Now, [Mserv]({{< ref "tyk-cloud#uploading-your-bundle" >}}), supports **multiple plugin bundles**, allowing greater customization and easier deployment of plugin configurations. Additionally, we added an **embedded product tour** to enhance the deployment experience, offering a guided walkthrough of Tyk Dashboard’s features, ideal for users familiarizing themselves with the platform during onboarding. For a comprehensive list of improvements and fixes, please check the detailed [changelog]({{< ref "#Changelog-v1.23.0">}}) below. -#### Breaking Changes - +### Breaking Changes There are no breaking changes in this release -#### Downloads +### Downloads - [latest version of Mserv](https://github.com/TykTechnologies/mserv/releases/latest) -#### Deprecations - +### Deprecations + There are no deprecations in this release -#### Changelog {#Changelog-v1.23.0} - +### Changelog {#Changelog-v1.23.0} -##### Added - +#### Added -##### Changed - +#### Changed -##### Fixed - +#### Fixed ##### Security Fixes - + - ## Further Information ### FAQ From 19f5696f027210dce69f027559b524f80ed89378 Mon Sep 17 00:00:00 2001 From: JoanCamosTyk <157505463+JoanCamosTyk@users.noreply.github.com> Date: Mon, 10 Feb 2025 12:40:58 +0100 Subject: [PATCH 3/8] [DX-1804] Update special-releases.md (#5919) --- .../release-notes/special-releases.md | 29 ++++++++++++++++++- 1 file changed, 28 insertions(+), 1 deletion(-) diff --git a/tyk-docs/content/developer-support/release-notes/special-releases.md b/tyk-docs/content/developer-support/release-notes/special-releases.md index 8bb2182a62..f92cf9edbd 100644 --- a/tyk-docs/content/developer-support/release-notes/special-releases.md +++ b/tyk-docs/content/developer-support/release-notes/special-releases.md @@ -129,7 +129,7 @@ As part of the release of the new Gateway LTS version we will commit to showing 1. **Recommended releases** - To ensure you get the most out of the latest Tyk experience, we'll provide information on which versions of different components across the entire stack you need. 2. **Backwards Compatibility** - We'll provide information on what components and versions remain backward compatible with the new Tyk Dashboard and Tyk Gateway versions. -Our next LTS version will be announced in April 2024. +Our next LTS version will be announced in April 2025. + +- `Headers`: this is an object of string arrays, and represents the current state of the request header; this object cannot be modified directly, but can be used to read header data +- `SetHeaders`: this is a key-value map that will be set in the header when the middleware returns the object; existing headers will be overwritten and new headers will be added +- `DeleteHeaders`: any header name that is in this list will be deleted from the outgoing request; note that `DeleteHeaders` happens before `SetHeaders` +- `Body`: this represents the body of the request, if you modify this field it will overwrite the request +- `URL`: this represents the path portion of the outbound URL, you can modify this to redirect a URL to a different upstream path +- `AddParams`: you can add parameters to your request here, for example internal data headers that are only relevant to your network setup +- `DeleteParams`: these parameters will be removed from the request as they pass through the middleware; note `DeleteParams` happens before `AddParams` +- `ReturnOverrides`: values stored here are used to stop or halt middleware execution and return an error code +- `IgnoreBody`: if this parameter is set to `true`, the original request body will be used; if set to `false` the `Body` field will be used (`false` is the default behavior) +- `Method`: contains the HTTP method (`GET`, `POST`, etc.) +- `RequestURI`: contains the request URI, including the query string, e.g. `/path?key=value` +- `Scheme`: contains the URL scheme, e.g. `http`, `https` + + +##### Using `ReturnOverrides` + +If you configure values in `request.ReturnOverrides` then Tyk will terminate the request and provide a response to the client when the function completes. The request will not be proxied to the upstream. + +The response will use the parameters configured in `ReturnOverrides`: + +- `ResponseCode` +- `ResponseBody` +- `ResponseHeaders` + +In this example, if the condition is met, Tyk will return `HTTP 403 Access Denied` with the custom header `"X-Error":"the-condition"`: + +```js +var testJSVMData = new TykJS.TykMiddleware.NewMiddleware({}); + +testJSVMData.NewProcessRequest(function(request, session, config) { + // Logic to determine if the request should be overridden + if (someCondition) { + request.ReturnOverrides.response_code = 403; + request.ReturnOverrides.response_body = "Access Denied"; + request.ReturnOverrides.headers = {"X-Error": "the-condition"}; + // This stops the request from proceeding to the upstream + } + return testJSVMData.ReturnData(request, session.meta_data); +}); +``` + +##### The virtual endpoint `request` object {#VirtualEndpoint-Request} + +For [virtual endpoint]({{< ref "advanced-configuration/compose-apis/virtual-endpoints" >}}) functions the structure of a Javascript `request` object is: + +```typescript +class VirtualEndpointRequest { + Body: string = ""; + Headers: { [key: string]: string[] } = {}; + Params: { [key: string]: string[] } = {}; + Scheme: string = ""; + URL: string = ""; +} +``` + +- `Body`: HTTP request body, e.g. `""` +- `Headers`: HTTP request headers, e.g. `"Accept": ["*/*"]` +- `Params`: Decoded query and form parameters, e.g. `{ "confirm": ["true"], "userId": ["123"] }` +- `Scheme`: The scheme of the URL ( e.g. `http` or `https`) +- `URL`: The full URL of the request, e.g `/vendpoint/anything?user_id=123\u0026confirm=true` + +
+ +{{< note success >}} +**Note** + +Each query and form parameter within the request is stored as an array field in the `Params` field of the request object. + +Repeated parameter assignments are appended to the corresponding array. For example, a request against `/vendpoint/anything?user_id[]=123&user_id[]=234` would result in a Javascript request object similar to that shown below: + +```javascript +const httpRequest = { + Headers: { + "Accept": ["*/*"], + "User-Agent": ["curl/8.1.2"] + }, + Body: "", + URL: "/vendpoint/anything?user_id[]=123\u0026user_id[]=234", + Params: { + "user_id[]": ["123", "234"] + }, + Scheme: "http" +}; +``` +{{< /note >}} + +#### The `session` object + +Tyk uses an internal [session object]({{< ref "getting-started/key-concepts/what-is-a-session-object" >}}) to handle the quota, rate limits, access allowances and auth data of a specific key. JS middleware can be granted access to the session object but there is also the option to disable it as deserialising it into the JSVM is computationally expensive and can add latency. Other than the `meta_data` field, the session object itself cannot be directly edited as it is crucial to the correct functioning of Tyk. + +##### Limitations + +- Custom JS plugins at the [pre-]({{< ref "api-management/plugins/plugin-types#request-plugins" >}}) stage do not have access to the session object (as it has not been created yet) +- When scripting for Virtual Endpoints, the `session` data will only be available to the JS function if enabled in the middleware configuration. + +##### Sharing data between middleware using the `session` object + +For different middleware to be able to transfer data between each other, the session object makes available a `meta_data` key/value field that is written back to the session store (and can be retrieved by the middleware down the line) - this data is permanent, and can also be retrieved by the REST API from outside of Tyk using the `/tyk/keys/` method. + +{{< note success >}} +**Note** + +A new JSVM instance is created for *each* API that is managed. Consequently, inter-API communication is not possible via shared methods, since they have different bounds. However, it *is* possible using the session object if a key is shared across APIs. +{{< /note >}} + +#### The `config` object + +The third Tyk data object that is made available to the script running in the JSVM contains data from the API Definition. This is read-only and cannot be modified by the JS function. The structure of this object is: + +- `APIID`: the unique identifier for the API +- `OrgID`: the organization identifier +- `config_data`: custom attributes defined in the API description + +##### Adding custom attributes to the API Definition + +When working with Tyk OAS APIs, you can add custom attributes in the `data` object in the `x-tyk-api-gateway.middleware.global.pluginConfig` section of the API definition, for example: + +```json {linenos=true, linenostart=1} +{ + "x-tyk-api-gateway": { + "middleware": { + "global": { + "pluginConfig": { + "data": { + "enabled": true, + "value": { + "foo": "bar" + } + } + } + } + } + } +} +``` + +When working with Tyk Classic APIs, you simply add the attributes in the `config_data` object in the root of the API definition: + +```json {linenos=true, linenostart=1} +{ + "config_data": { + "foo": "bar" + } +} +``` + +#### Underscore.js Library + +In addition to our Tyk JavaScript API functions, you also have access to all the functions from the [underscore](http://underscorejs.org) library. + +Underscore.js is a JavaScript library that provides a lot of useful functional programming helpers without extending any built-in objects. Underscore provides over 100 functions that support your favorite functional helpers: + +- map +- filter +- invoke + +There are also more specialized goodies, including: + +- function binding +- JavaScript templating +- creating quick indexes +- deep equality testing + +### Example + +In this basic example, we show the creation and initialisation of a middleware object. Note how the three Tyk data objects (`request`, `session`, `config`) are made available to the function and the two objects that are returned from the function (in case the external objects need to be updated). + +```js {linenos=true, linenostart=1} +/* --- sampleMiddleware.js --- */ + +// Create new middleware object +var sampleMiddleware = new TykJS.TykMiddleware.NewMiddleware({}); + +// Initialise the object with your functionality by passing a closure that accepts +// two objects into the NewProcessRequest() function: +sampleMiddleware.NewProcessRequest(function(request, session, config) { + log("This middleware does nothing, but will print this to your terminal.") + + // You MUST return both the request and session metadata + return sampleMiddleware.ReturnData(request, session.meta_data); +}); +``` + +## JavaScript API + +This system API provides access to resources outside of the JavaScript Virtual Machine sandbox, the ability to make outbound HTTP requests and access to the key management REST API functions. + +Embedded JavaScript interpreters offer the bare bones of a scripting language, but not necessarily the functions that you would expect, especially with JavaScript, where objects such as `XMLHttpRequest()` are a given. However, those interfaces are actually provided by the browser / DOM that the script engine are executing in. In a similar vein, we have included a series of functions to the JSVM for convenience and give the interpreter more capability. + +This list is regularly revised and any new suggestions should be made in our [Github issue tracker](https://github.com/TykTechnologies/tyk/issues). + +Below is the list of functions currently provided by Tyk. + +- `log(string)`: Calling `log("this message")` will cause Tyk to log the string to Tyk's default logger output in the form `JSVM Log: this message` as an INFO statement. This function is especially useful for debugging your scripts. It is recommended to put a `log()` call at the end of your middleware and event handler module definitions to indicate on load that they have been loaded successfully - see the [example scripts](https://github.com/TykTechnologies/tyk/tree/master/middleware) in your Tyk installation `middleware` directory for more details. +- `rawlog(string)`: Calling `rawlog("this message")` will cause Tyk to log the string to Tyk's default logger output without any additional formatting, like adding prefix or date. This function can be used if you want to have own log format, and parse it later with custom tooling. +- `b64enc` - Encode string to base64 +- `b64dec` - Decode base64 string +- `TykBatchRequest` this function is similar to `TykMakeHttpRequest` but makes use of the Tyk Batch API. See the Batch Requests section of the [Tyk Gateway API]({{< ref "tyk-gateway-api" >}}) for more details. +- `TykMakeHttpRequest(JSON.stringify(requestObject))`: This method is used to make an HTTP request, requests are encoded as JSON for deserialisation in the min binary and translation to a system HTTP call. The request object has the following structure: + +```js +newRequest = { + "Method": "POST", + "Body": JSON.stringify(event), + "Headers": {}, + "Domain": "http://foo.com", + "Resource": "/event/quotas", + "FormData": {"field": "value"} +}; +``` + +{{< note success >}} +**Note** + +If you want to include querystring values, add them as part of the `Domain` property. +{{< /note >}} + +Tyk passes a simplified response back which looks like this: + +```go +type TykJSHttpResponse struct { + Code int + Body string + Headers map[string][]string +} +``` + +The response is JSON string encoded, and so will need to be decoded again before it is usable: + +```js +usableResponse = JSON.parse(response); +log("Response code: " + usableResponse.Code); +log("Response body: " + usableResponse.Body); +``` + +This method does not execute asynchronously, so execution will block until a response is received. + +### Working with the key session object + +To work with the key session object, two functions are provided: `TykGetKeyData` and `TykSetKeyData`: + +- `TykGetKeyData(api_key, api_id)`: Use this method to retrieve a [session object]({{< ref "getting-started/key-concepts/what-is-a-session-object" >}}) for the key and the API provided: + + ```js + // In an event handler, we can get the key idea from the event, and the API ID from the context variable. + var thisSession = JSON.parse(TykGetKeyData(event.EventMetaData.Key, context.APIID)) + log("Expires: " + thisSession.expires) + ``` + +- `TykSetKeyData(api_key, api_id)`: Use this method to write data back into the Tyk session store: + + ```js + // You can modify the object just like with the REST API + thisSession.expires = thisSession.expires + 1000; + + // Use TykSetKeyData to set the key data back in the session store + TykSetKeyData(event.EventMetaData.Key, JSON.stringify(thisSession)); + ``` + +All of these methods are described in functional examples in the Tyk `middleware/` and `event_handlers/` folders. + +## Installing Middleware on Tyk Self-Managed + +In some cases middleware references can't be directly embedded in API Definitions (for example, when using the Tyk Dashboard in an Self-Managed installation). However, there is an easy way to distribute and enable custom middleware for an API in a Tyk node by adding them as a directory structure. + +Tyk will load the middleware plugins dynamically on host-reload without needing a direct reference to them in the API Definition. + +The directory structure should look like this: + +```text +middleware + / {API Id} + / pre + / {middlewareObject1Name}.js + / {middlewareObject2Name}.js + / post + / {middlewareObject1Name}_with_session.js + / {middlewareObject2Name}.js +``` + +Tyk will check for a folder that matches the `API Id` being loaded, and then load the `pre` and `post` middleware from the respective directories. + +{{< note success >}} +**Note** + +The filename MUST match the object to be loaded exactly. +{{< /note >}} + +If your middleware requires session injection, then append `_with_session` to the filename. + +### Enable the JSVM + +Before you can use Javascript Middleware you will need to enable the JSVM. + +You can do this by setting `enable_jsvm` to `true` in your `tyk.conf` file. + +## Installing Middleware on Tyk Hybrid + +In some cases middleware references can't be directly embedded in API Definitions (for example, when using the dashboard in a Hybrid install). However, there is an easy way to distribute and enable custom middleware for an API on a Tyk node. + +A second method of loading API Definitions in Tyk nodes is to add them as a directory structure in the Tyk node. Tyk will load the middleware plugins dynamically on host-reload without needing a direct reference to them in the API Definition. + +The directory structure looks as follows: + +```text +middleware + / {API Id} + / pre + / {middlewareObject1Name}.js + / {middlewareObject2Name}.js + / post + / {middlewareObject1Name}_with_session.js + / {middlewareObject2Name}.js +``` + +Tyk will check for a folder that matches the `{API Id}` being loaded, and then load the `pre` and `post` middleware from the respective folders. + +{{< note success >}} +**Note** + +The filename MUST match the object to be loaded exactly. +{{< /note >}} + +If your middleware requires session injection, then append `_with_session` to the filename. + +### Enable the JSVM + +Before you can use Javascript Middleware you will need to enable the JSVM + +You can do this by setting `enable_jsvm` to `true` in your `tyk.conf` file. + +## Installing Middleware on Tyk OSS + +In order to activate middleware when using Tyk OSS or when using a file-based setup, the middleware needs to be registered as part of your API Definition. Registration of middleware components is relatively simple. + +{{< note success >}} +**Note** + +It is important that your object names are unique. +{{< /note >}} + +{{< note success >}} +**Note** + +This functionality may change in subsequent releases. +{{< /note >}} + +### Enable the JSVM + +Before you can use Javascript Middleware you will need to enable the JSVM + +You can do this by setting `enable_jsvm` to `true` in your `tyk.conf` file. + +Adding the middleware plugin is as simple as adding it to your definition file in the middleware sections: + +```json +... +"event_handlers": {}, +"custom_middleware": { + "pre": [ + { + "name": "sampleMiddleware", + "path": "middleware/sample.js", + "require_session": false + } + ], + "post": [ + { + "name": "sampleMiddleware", + "path": "middleware/sample.js", + "require_session": false + } + ] +}, +"enable_batch_request_support": false, +... +``` + +As you can see, the parameters are all dynamic, so you will need to ensure that the path to your middleware is correct. The configuration sections are as follows: + +- `pre`: Defines a list of custom middleware objects to run *in order* from top to bottom. That will be executed *before* any authentication information is extracted from the header or parameter list of the request. Use middleware in this section to pre-process a request before feeding it through the Tyk middleware. + +- `pre[].name`: The name of the middleware object to call. This is case sensitive, and **must** match the name of the middleware object that was created, so in our example - we created `sampleMiddleware` by calling: + + `var sampleMiddleware = new TykJS.TykMiddleware.NewMiddleware({});` + +- `pre[].path`: The path to the middleware component, this will be loaded into the JSVM when the API is initialised. This means that if you reload an API configuration, the middleware will also be re-loaded. You can hot-swap middleware on reload with no service interruption. + +- `pre[].require_session`: Irrelevant for pre-processor middleware, since no auth data has been extracted by the authentication middleware, it cannot be made available to the middleware. + +- `post`: Defines a list of custom middleware objects to run *in order* from top to bottom. That will be executed *after* the authentication, validation, throttling, and quota-limiting middleware has been executed, just before the request is proxied upstream. Use middleware in this section to post-process a request before sending it to your upstream API. + +- `post[].name`: The name of the middleware object to call. This is case sensitive, and **must** match the name of the middleware object that was created, so in our example - we created `sampleMiddleware` by calling: + + `var sampleMiddleware = new TykJS.TykMiddleware.NewMiddleware({});` + +- `post[].path`: The path to the middleware component, this will be loaded into the JSVM when the API is initialised. This means that if you reload an API configuration, the middleware will also be re-loaded. You can hot-swap middleware on reload with no service interruption. + +- `post[].require_session`: Defaults to `false`, if you require access to the session object, it will be supplied as a `session` variable to your middleware processor function. + +## WAF (OSS) ModSecurity Plugin example + +Traditionally, a Web Application Firewall (WAF) would be the first layer requests would hit, before reaching the API gateway. This is not possible if the Gateway has to terminate SSL, for things such as mTLS. + +So what do you do if you still want to run your requests through a WAF to automatically scan for malicious action? We incorporate a WAF as part of the request lifecycle by using Tyk's plugin architecture. + +### Prerequisites + +* Already running Tyk - Community Edition or Pro +* Docker, to run the WAF + +### Disclaimer + +This is NOT a production ready plugin because + +* The JavaScript plugin creates a new connection with the WAF for every request +* The request is not sent over SSL +* The WAF is only sent the query params for inspection. + +For higher performance, the plugin could be written in Golang, and a connection pool would be opened and maintained over SSL + +### Steps for Configuration + +1. **Turn JSVM on your `tyk.conf` at the root level:** + + Turn on JSVM interpreter to allow Tyk to run JavaScript plugins. + + ``` + "enable_jsvm": true + ``` + +2. **Place the JavaScript plugin on Tyk file system** + + Copy the JS Plugin as a local .js file to the Gateway's file system. + + From the Gateway root, this will download the plugin called `waf.js` into the `middleware` directory: + ``` + curl https://raw.githubusercontent.com/TykTechnologies/custom-plugins/master/plugins/js-pre-post-waf/waf.js | cat > middleware/waf.js + ``` + + (Instructions) + If you are running Tyk in Docker, you can get into Tyk Gateway with `docker exec` + ``` + $ docker ps | grep gateway + 670039a3e0b8 tykio/tyk-gateway:latest "./entrypoint.sh" 14 minutes ago Up 14 minutes 0.0.0.0:8080->8080/tcp tyk-demo_tyk-gateway_1 + + ## copy container name or ID + $ docker exec -it 670039a3e0b8 bash + + ## Now SSH'd into Tyk Gateway container and can perform curl + root@670039a3e0b8:/opt/tyk-gateway# ls + + apps entrypoint.sh install middleware templates tyk-gateway.pid tyk.conf.example + coprocess event_handlers js policies tyk tyk.conf utils + + ## Download the plugin + root@670039a3e0b8:/opt/tyk-gateway# curl https://raw.githubusercontent.com/TykTechnologies/custom-plugins/master/plugins/js-pre-post-waf/waf.js | cat > middleware/waf.js + + % Total % Received % Xferd Average Speed Time Time Time Current + Dload Upload Total Spent Left Speed + 100 1125 100 1125 0 0 3906 0 --:--:-- --:--:-- --:--:-- 3975 + + ``` + + [waf.js source](https://raw.githubusercontent.com/TykTechnologies/custom-plugins/master/plugins/js-pre-post-waf/waf.js) + +3. **Import API definition into Tyk** + + Copy the following Tyk API definition and import it into your environment. + + [API Definition JSON](https://raw.githubusercontent.com/TykTechnologies/custom-plugins/master/plugins/js-pre-post-waf/apidef.json) + + Here's the important section which adds the plugin to the request lifecycle for this API: + ```{.json} + "custom_middleware": { + "pre": [ + { + "name": "Waf", + "path": "./middleware/waf.js" + } + ], + ``` + + ##### How to Import? + [Tyk Pro](https://tyk.io/docs/tyk-configuration-reference/import-apis/#import-apis-via-the-dashboard) + + [Tyk CE](https://tyk.io/docs/try-out-tyk/tutorials/create-api/) + +4. **Run WAF ModSecurity Using Docker** + + First run ModSecurity with the popular [Core RuleSet](https://coreruleset.org/) in Docker + ``` + $ docker run -ti -p 80:80 -e PARANOIA=1 --rm owasp/modsecurity-crs:v3.0 + ``` + + Open a second terminal and curl it + ``` + $ curl localhost + + hello world + ``` + + We should see the request show in the WAF server: + + ``` + 172.17.0.1 - - [30/Jun/2020:00:56:42 +0000] "GET / HTTP/1.1" 200 12 + ``` + + Now try a dirty payload: + ``` + $ curl 'localhost/?param=">' + + + + 403 Forbidden + +

Forbidden

+

You don't have permission to access / + on this server.
+

+ + ``` + + Our WAF catches the response and returns a `403`. + + + Now we try through Tyk. + + ``` + ## Clean requests, should get response from upstream's IP endpoint + $ curl localhost:8080/waf/ip + + { + "origin": "172.30.0.1, 147.253.129.30" + } + + ## WAF will detect malicious payload and instruct Tyk to deny + $ curl 'localhost:8080/waf/ip?param="> + { + "error": "Bad request!" + } + ``` diff --git a/tyk-docs/content/api-management/plugins/overview.md b/tyk-docs/content/api-management/plugins/overview.md new file mode 100644 index 0000000000..672892e368 --- /dev/null +++ b/tyk-docs/content/api-management/plugins/overview.md @@ -0,0 +1,1214 @@ +--- +title: "Custom Plugins" +date: 2025-01-10 +tags: [] +description: "" +keywords: [] +aliases: + - /plugins + - /plugins/rich-plugins/plugin-bundles + - /plugins/how-to-serve/plugin-bundles + - /plugins/how-to-serve + - /customise-tyk/plugins + - /plugins/get-started-selfmanaged/deploy + - /plugins/get-started-selfmanaged/get-started + - /plugins/get-started-selfmanaged/run + - /plugins/get-started-selfmanaged/test + - /plugins/get-started-plugins + - /plugins/tutorials/quick-starts/go/quickstart + - /plugins/tutorials/quick-starts/go/dashboard + - /plugins/tutorials/quick-starts/go/open-source + - /plugins/plugin-hub + - /product-stack/tyk-gateway/advanced-configurations/plugins/api-config/overview + - /product-stack/tyk-gateway/advanced-configurations/plugins/api-config/oas + - /product-stack/tyk-gateway/advanced-configurations/plugins/api-config/classic + - /plugins/how-to-serve-plugins + - /plugins/how-to-serve-plugins/plugin-bundles + - /product-stack/tyk-gateway/advanced-configurations/plugins/bundles/oas + - /product-stack/tyk-gateway/advanced-configurations/plugins/bundles/classic + - /product-stack/tyk-gateway/advanced-configurations/plugins/bundles/bundle-cli + - /plugins/supported-languages +--- + +## Introduction + +Plugins can be used to customize and enhance the capabilities of your APIs through integration with external services and databases to perform operations such as data transformation, custom authentication, logging and monitoring etc. + +When Tyk receives an API request, it works through a [chain]({{< ref "middleware-execution-order" >}}) of processing *middleware* that is configured using the API definition. There are a large number of built-in middleware in the processing chain that are dedicated to performing [client authentication]({{< ref "/api-management/client-authentication" >}}), [request transformation]({{< ref "advanced-configuration/transform-traffic" >}}), [caching]({{< ref "basic-config-and-security/reduce-latency/caching" >}}) and many other processes before proxying the request to the upstream. + +Tyk's custom plugin facility provides a powerful and flexible way to extend the middleware chain. It allows API developers to write custom middleware, in various programming languages, that can perform additional processing of requests and responses. + +For example, a custom authentication scheme can be implemented and executed on API requests, custom plugins can be used to provide integration with external services and databases, or additional processing can be performed on the response returned from the upstream. + +There are several different stages of the [API request lifecycle]({{< ref "/concepts/middleware-execution-order" >}}) where custom plugins can be attached (or *hooked*) into the middleware chain allowing significant customization to meet your specific requirements. + +Custom plugins are usually referred to by the location where they can be *hooked* into the middleware processing chain as follows: + +1. [Pre (Request)]({{< ref "api-management/plugins/plugin-types#request-plugins" >}}) +2. [Authentication]({{< ref "api-management/plugins/plugin-types#authentication-plugins" >}}) +3. [Post-Auth (Request)]({{< ref "api-management/plugins/plugin-types#request-plugins" >}}) +4. [Post (Request)]({{< ref "api-management/plugins/plugin-types#request-plugins" >}}) +5. [Response]({{< ref "api-management/plugins/plugin-types#response-plugins" >}}) +6. [Analytics (Response)]({{< ref "api-management/plugins/plugin-types#analytics-plugins" >}}) + +## How Plugin Works + +The diagram below illustrates a high level architectural overview for how Tyk Gateway interacts with plugins. + +{{< img src="/img/plugins/plugins_overview.svg" width="500px" alt="plugins overview" >}} + +From the above illustration it can be seen that: + +1. The client sends a request to an API served by Tyk Gateway. +2. Tyk processes the request and forwards it to one or more plugins implemented and configured for that API. +3. A plugin performs operations (e.g., custom authentication, data transformation). +4. The processed request is then returned to Tyk Gateway, which forwards it upstream. +5. Finally, the upstream response is sent back to the client. + +## Types of Plugin + +Tyk supports four types of plugins: + +1. **[Request Plugin]({{< ref "" >}})** +2. **[Authentication Plugin]({{< ref "" >}})** +3. **[Response Plugin]({{< ref "" >}})** +4. **[Analytics Plugin]({{< ref "" >}})** + +To know more about plugin types and it's advanced configuration, refer the following [docs]({{< ref "" >}}). + +## Getting Started + +This section takes you through the process of running and building a quickstart **Go plugin**, included within Tyk's [getting started](https://github.com/TykTechnologies/custom-go-plugin) repository. Go plugins are the recommended plugin type and suitable for most use cases. + +##### Expected outcome + +At the end of this process you should have a Tyk Gateway or Tyk Self-Managed environment running locally, with a simple Go plugin executing on each API request. For each reponse to an API request the example plugin will inject a *Foo* header, with a value of *Bar*. + +##### Prerequisites + +- [Docker](https://docs.docker.com/get-docker/) +- [Docker-compose](https://docs.docker.com/compose/install/) +- [Tyk license](https://tyk.io/sign-up/#self) (if using Self-Managed Tyk, which will make the process easier via UI) +- [Make](https://www.gnu.org/software/make) +- OSX (Intel) -> Not a prerequisite, though these steps are tested on OSX Intel/ARM + +##### Before you begin + +Please clone the [getting started](https://github.com/TykTechnologies/custom-go-plugin) respository. + +```bash +git clone https://github.com/TykTechnologies/custom-go-plugin +cd custom-go-plugin +``` + +##### Choose your environment + +{{< grid >}} +{{< badge read="15 mins" imageStyle="object-fit:contain" href="plugins/tutorials/quick-starts/go/dashboard" image="/img/logos/tyk-logo-selfmanaged.svg" alt="Dashboard">}} +Dashboard Tutorial +{{< /badge >}} + +{{< badge read="15 mins" imageStyle="object-fit:contain" href="plugins/tutorials/quick-starts/go/open-source" image="/img/logos/tyk-logo-selfmanaged.svg" alt="Tyk OSS Gateway">}} +Tyk OSS Gateway Tutorial +{{< /badge >}} +{{< /grid >}} + + +### Dashboard Plugins + +This quick start explains how to run the [getting started](https://github.com/TykTechnologies/custom-go-plugin) Go plugin within Tyk Dashboard. + +**Estimated time**: 10-15 minutes + +In this tutorial you will learn how to: + +1. Add your Tyk license. +2. Bootstrap the Tyk Dashboard environment. +3. Login to Tyk Dashboard. +4. View the pre-configured API. +5. Test the plugin. +6. View the analytics. +7. Next steps. + +**Steps for Configuration:** + +1. **Add your Tyk license** + + Create and edit the file `.env` with your Tyk Dashboard license key + + ```console + # Make a copy of the example .env file for the Tyk-Dashboard + cp .env.example .env + ``` + +2. **Bootstrap the getting started example** + + run the `make` command: + + ```bash + make + ``` + + This will take a few minutes to run as it compiles the plugin for the first time and downloads all the necessary Docker images. + +3. **Log in to Tyk Dashboard** + + Log on to the Tyk Dashboard on `http://localhost:3000` using the following Bootstrapped credentials: + ``` + demo@tyk.io + ``` + and password: + ``` + topsecretpassword + ``` + + Note: these are editable in `.env.example` + +4. **View the pre-configured API** + + Once you're logged on to the Tyk Dashboard, navigate to the *APIs* screen. + + You'll see a sample *Httpbin* API. Let's click into it for more details. + + Click on *VIEW RAW DEFINITION*. Note the *custom_middleware* block is filled out, injecting the compiled example Go plugin into the API. + +5. **Test the plugin** + + Let's send an API request to the API Gateway so it can reverse proxy to our API. + + ```terminal + curl localhost:8080/httpbin/get + ``` + + Yields the response: + ``` + { + "args": {}, + "headers": { + "Accept": "*/*", + "Accept-Encoding": "gzip", + "Foo": "Bar", + "Host": "httpbin.org", + "User-Agent": "curl/7.79.1", + "X-Amzn-Trace-Id": "Root=1-63f78c47-51e22c5b57b8576b1225984a" + }, + "origin": "172.26.0.1, 99.242.70.243", + "url": "http://httpbin.org/get" + } + ``` + + Note, we see a *Foo:Bar* HTTP Header was injected by our Go plugin and echoed back to us by the Httpbin mock server. + +6. **View the analytics** + + Navigate to the Dashboard's various *API Usage Data* to view analytics on the API request! + +### Open-Source Plugins + +This quick start guide will explain how to run the [getting started](https://github.com/TykTechnologies/custom-go-plugin) Go plugin using the Tyk OSS Gateway. + +**Steps for Configuration:** + +1. **Bootstrap the getting started example** + + Please run the following command from within your newly cloned directory to run the Tyk Stack and compile the sample plugin. This will take a few minutes as we have to download all the necessary dependencies and docker images. + + ```bash + make up-oss && make build + ``` + +2. **Test the plugin** + + Let's test the plugin by sending an API request to the pre-configured API definition: + + ``` + curl localhost:8080/httpbin/get + ``` + + Response: + ``` + { + "args": {}, + "headers": { + "Accept": "*/*", + "Accept-Encoding": "gzip", + "Foo": "Bar", + "Host": "httpbin.org", + "User-Agent": "curl/7.79.1" + }, + "origin": "172.28.0.1, 99.242.70.243", + "url": "http://httpbin.org/get" + } + ``` + + We've sent an API request to the Gateway. We can see that the sample custom plugin has injected an HTTP header with a value of *Foo:Bar*. This header was echoed back in the Response Body via the mock Httpbin server. + + The `./tyk/scripts/bootstrap-oss.sh` script creates an API definition that includes the custom plugin. + +3. **View the analytics** + + We can see that Tyk Pump is running in the background. Let's check the logs after sending the API request: + + ``` + docker logs custom-go-plugin_tyk-pump_1 + ``` + + Output: + ``` + time="Feb 23 16:29:27" level=info msg="Purged 1 records..." prefix=stdout-pump + {"level":"info","msg":"","time":"0001-01-01T00:00:00Z","tyk-analytics-record":{"method":"GET","host":"httpbin.org","path":"/get","raw_path":"/get","content_length":0,"user_agent":"curl/7.79.1","day":23,"month":2,"year":2023,"hour":16,"response_code":200,"api_key":"00000000","timestamp":"2023-02-23T16:29:27.53328605Z","api_version":"Non Versioned","api_name":"httpbin","api_id":"845b8ed1ae964ea5a6eccab6abf3f3de","org_id":"","oauth_id":"","request_time":1128,"raw_request":"...","raw_response":"...","ip_address":"192.168.0.1","geo":{"country":{"iso_code":""},"city":{"geoname_id":0,"names":null},"location":{"latitude":0,"longitude":0,"time_zone":""}},"network":{"open_connections":0,"closed_connections":0,"bytes_in":0,"bytes_out":0},"latency":{"total":1128,"upstream":1111},"tags":["key-00000000","api-845b8ed1ae964ea5a6eccab6abf3f3de"],"alias":"","track_path":false,"expireAt":"2023-03-02T16:29:27.54271855Z","api_schema":""}} + ``` + + As we can see, when we send API requests, the Tyk Pump will scrape them from Redis and then send them to a persistent store as configured in the Tyk Pump env file. + + In this example, we've configured a simple `STDOUT` Pump where the records will be printed to the Standard OUT (docker logs!) + +## API Configuration + +This page provides an overview on how to register one or more custom plugins to be executed at different stages or [hooks]({{< ref "api-management/plugins/plugin-types#plugin-and-hook-types" >}}) in the API request/response lifecycle. If you wish to learn how to register custom plugins to be executed on the traffic logs generated by the Gateway please refer to the [analytics plugins]({{< ref "api-management/plugins/plugin-types#analytics-plugins" >}}) page. + +If you need fine-grained control at the endpoint level then it is also possible to configure [per-endpoint plugins]({{< ref "api-management/plugins/plugin-types#per-endpoint-custom-plugins" >}}). These are custom Golang plugins that are triggered at the end of the request processing chain before API-level *Post* plugins are executed. + +--- + +### Introduction + +There are three locations where Tyk Gateway can find plugin functions: + +1. **gRPC plugins**: Plugin functions are implemented by a gRPC server with the associated configuration specified with the API definition. For further details on how to configure gRPC plugins, please refer to our [gRPC]({{< ref "api-management/plugins/rich-plugins#overview-1" >}}) documentation. +2. **Local plugins**: Plugins are implemented by functions within source code files located on the Gateway's file system. The API Definition allows the source code file path and function name to be configured for each plugin. For further details read on. +3. **Plugin bundles**: The plugin source code and configuration are bundled into a zip file that is served by a remote web server. For further details see the [plugin bundles]({{< ref "api-management/plugins/overview#plugin-bundles" >}}) page. + +### Plugin configuration + +Each plugin for an API can be configured within the API Definition with the following details: + +| Property | Description | +|-------|-------------| +| `Enabled` | When true, the plugin is activated | +| `Name` | A name used to identify the plugin | +| `Path` | The path to the source code file on the Tyk Gateway file system | +| `Function name` | The name of the function that implements the plugin. The function should exist within the source code file referenced in `path` | +| `Raw body only` | When set to true, this flag indicates that only the raw request body should be processed | +| `Require session state`| When set to true, Tyk Gateway will serialize the request session state and pass it as an argument to the function that implements the plugin in the target language. This is applicable to Post, Response, and Authentication hooks only | + +--- + +### Language configuration + +For local and bundle plugins a [plugin driver]({{< ref "api-management/plugins/overview#plugin-driver-names" >}}) is configured to specify the plugin implementation language. If using gRPC plugins a `grpc` plugin driver should be used to instruct Tyk to request execution of plugins from within a gRPC server that is external to the Tyk process. This offers additional language support since Tyk can integrate with a gRPC server that is implemented using any supported [gRPC language](https://grpc.io/docs/). + +For a given API it is not possible to mix the implementation language for the plugin types: Pre, Authentication, Post, Post Authentication and Response plugins. For example, it is not possible to implement a pre request plugin in *Go* and also implement a post request plugin in *Python* for the same API. + +### Tyk OAS APIs + +An API can be configured so that one or more of its associated plugins can execute at different phases of the request / response life cycle. Each plugin configuration serves to identify the plugin source file path and the name of the corresponding function, triggered at each request / response lifecycle stage. + +This guide explains how to configure plugins for Tyk OAS APIs within the [Tyk OAS API definition](#tyk-oas-apidef) or via the [API designer](#tyk-oas-dashboard) in Tyk Dashboard. + +If you’re using the legacy Tyk Classic APIs, then check out the [Tyk Classic]({{< ref "api-management/plugins/overview#tyk-classic-apis" >}}) page. + +#### Using API Definition {#tyk-oas-apidef} + +The `x-tyk-api-gateway.middleware.global` section is used to configure plugins in a Tyk OAS API. It contains a `pluginConfig` section and a list of plugins for each phase of the API request / response lifecycle. + +The `pluginConfig` section contains the `driver` parameter that is used to configure the plugin implementation [language]({{< ref "api-management/plugins/overview#plugin-driver-names" >}}): + +```yaml +"pluginConfig": { + "driver": "goplugin" +} +``` + +Within the `x-tyk-api-gateway.middleware.global` section, keyed lists of plugins can be configured for each phase of the API request / response lifecycle described in the table below: + +| Phase | Description | Config Key | +| ----- | --- | ---- | +| Pre | Executed at the start of the request processing chain | `prePlugins` | +| Post Auth | Executed after the requester has been authenticated | `postAuthenticationPlugins` | +| Post | Executed at the end of the request processing chain | `postPlugins` | +| Response | Occurs after the main request processing but before the response is sent. | `responsePlugins` | + +Each plugin configuration can have the following fields configured: + +- `enabled`: When true, enables the plugin. +- `functionName`: The name of the function that implements the plugin within the source file. +- `path`: The path to the plugin source file. +- `rawBodyOnly`: When true, indicates that only the raw body should be processed. +- `requireSession`: When true, indicates that session metadata will be available to the plugin. This is applicable only for post, post authentication and response plugins. + +For example a Post Authentication plugin would be configured within a `postAuthenticationPlugins` list as shown below: + +```yaml +"postAuthenticationPlugins": [ + { + "enabled": true, + "functionName": "post_authentication_func", + "path": "/path/to/plugin1.so", + "rawBodyOnly": true, + "requireSession": true + } +] +``` + +An full example is given below to illustrate how to set up plugins for different phases of the request / response lifecycle: + +```json {linenos=true, linenostart=1, hl_lines=["15-52"]} +{ + "x-tyk-api-gateway": { + "info": { + "dbId": "667962397f6de50001508ac4", + "id": "b4d8ac6e5a274d7c7959d069b47dc206", + "orgId": "6672f4377f6de50001508abf", + "name": "OAS APIs Plugins", + "state": { + "active": true, + "internal": false + } + }, + "middleware": { + "global": { + "pluginConfig": { + "driver": "goplugin" + }, + "postAuthenticationPlugins": [ + { + "enabled": true, + "functionName": "post_authentication_func", + "path": "/path/to/plugin1.so", + "rawBodyOnly": true, + "requireSession": true + } + ], + "postPlugins": [ + { + "enabled": true, + "functionName": "postplugin", + "path": "/path/to/plugin1.so", + "rawBodyOnly": true, + "requireSession": true + } + ], + "prePlugins": [ + { + "enabled": true, + "functionName": "pre-plugin", + "path": "/path/to/plugin1.so" + } + ], + "responsePlugins": [ + { + "enabled": true, + "functionName": "Response", + "path": "/path/to/plugin1.so", + "rawBodyOnly": true, + "requireSession": true + } + ] + } + } + } +} +``` + +In this example we can see that the plugin driver has been configured by setting the `driver` field to `goplugin` within the `pluginConfig` object. This configuration instructs Tyk Gateway that our plugins are implemented using Golang. + +We can also see that the following type of plugins are configured: + +- **Pre**: A plugin is configured within the `prePlugins` list. The plugin is enabled and implemented by function `pre-plugin` within the source file located at path `/path/to/plugin1.so`. +- **Post Authentication**: A plugin is configured within the `postAuthenticationPlugins` list. The plugin is enabled and implemented by function `post_authentication_func` within the source file located at path `/path/to/plugin1.so`. The raw request body and session metadata is available to the plugin. +- **Post**: A plugin is configured within the `responsePlugins` list. The plugin is enabled and implemented by function `postplugin` within the source file located at path `/path/to/plugin1.so`. The raw request body and session metadata is available to the plugin. +- **Response**: A plugin is configured within the `postPlugins` list. The plugin is enabled and implemented by function `Response` within the source file located at path `/path/to/plugin1.so`. The raw request body and session metadata is available to the plugin. + +The configuration above is a complete and valid Tyk OAS API Definition that you can use as a basis for trying out custom plugins. You will need to update the [driver]({{< ref "api-management/plugins/overview#plugin-driver-names" >}}) parameter to reflect the target language type of your plugins. You will also need to update the `path` and `functionName` parameters for each plugin to reflect the source code. + +#### Using API Designer {#tyk-oas-dashboard} + +Select your API from the list of *Created APIs* to reach the API designer and then follow these steps: + +1. **Configure plugin type and custom data** + + In the *Plugins Configuration* section, select the *Plugin Driver*, which tells Tyk which type of plugin to expect: Go, gRPC, JavaScript (OTTO), Lua or Python. + + You can configure custom data that will be made available to your plugin function as a JSON formatted object in the *Config Data* option. + + {{< img src="/img/plugins/plugins_oas_api_driver_options.png" alt="OAS API Plugins Driver Config" >}} + +2. **Configure the custom plugins** + + For each plugin that you wish to register with the API, click on the **Add Plugin** button to display a plugin configuration section: + + {{< img src="/img/plugins/plugins_oas_api_source_config.png" alt="OAS Plugins Config Section" >}} + + Complete the following fields: + + - `Function Name`: Enter the name of the function within your plugin code that Tyk should invoke. + - `Path`: Enter the path to the source file that contains the function that implements your plugin. + - `Raw Body Only`: Optionally, toggle the *Raw Body Only* switch to true when you do not wish to fill body in request or response object for your plugins. + +3. **Save the API** + + Select **Save API** to apply the changes to your API. + +### Tyk Classic APIs + +An API can be configured so that one or more of its associated plugins can execute at different phases of the request / response lifecycle. Each plugin configuration serves to identify the plugin source file path and the name of the corresponding function, triggered at each request / response lifecycle stage. + +This guide explains how to configure plugins for Tyk Classic APIs within the [Tyk Classic API definition](#tyk-classic-apidef) or via the [API designer](#tyk-classic-dashboard) in Tyk Dashboard. + +If you’re using the newer Tyk OAS APIs, then check out the [Tyk OAS]({{< ref "api-management/plugins/overview#tyk-oas-apis" >}}) page. + +#### Using API Definition {#tyk-classic-apidef} + +In Tyk Classic APIs, the *custom_middleware* section of the Tyk Classic API Definition is where you configure plugins that will run at different points during the lifecycle of an API request. + +This table illustrates the different phases of the API request lifecycle where custom plugins can be executed: + +| Phase | Description | Config | +| ----- | --- | ---- | +| Pre | Executed at the start of the request processing chain | `pre` | +| Auth | Executed during the authentication step | `auth_check` | +| Post Auth | Executed after the requester has been authenticated | `post_key_auth` | +| Post | Executed at the end of the request processing chain | `post` | +| Response | Executed on the response received from the upstream | `response` | + +This example configuration illustrates how to set up plugins for different phases of the request lifecycle: + +```json {linenos=true, linenostart=1} +{ + "custom_middleware": { + "pre": [ + { + "name": "PreHook1", + "path": "/path/to/plugin1.so", + "disabled": false, + "require_session": false, + "raw_body_only": false + } + ], + "auth_check": { + "name": "AuthCheck", + "path": "/path/to/plugin.so", + "disabled": false, + "require_session": false, + "raw_body_only": false + }, + "post_key_auth": [ + { + "name": "PostKeyAuth", + "path": "/path/to/plugin.so", + "disabled": false, + "require_session": false, + "raw_body_only": false + } + ], + "post": [ + { + "name": "PostHook1", + "path": "/path/to/plugin1.so", + "disabled": false, + "require_session": false, + "raw_body_only": false + }, + { + "name": "PostHook2", + "path": "/path/to/plugin2.so", + "disabled": false, + "require_session": false, + "raw_body_only": false + } + ], + "response": [ + { + "name": "ResponseHook", + "path": "/path/to/plugin.so", + "disabled": false, + "require_session": false, + "raw_body_only": false + } + ], + "driver": "goplugin" + } +} +``` + +In this example we can see that there are Golang custom authentication (`auth_check`), post authentication (`post_key_auth`), post, pre and response plugins configured. + +It can be seen that each plugin is configured with the specific function name and associated source file path of the file that contains the function. Furthermore, each lifecycle phase (except `auth`) can have a list of plugins configured, allowing for complex processing workflows. For example, you might develop one plugin for logging and another for modifying the request in the pre request phase. When multiple plugins are configured for a phase they will be executed in the order that they appear in the API definition. + +The `driver` configuration parameter describes the plugin implementation language. Please refer to the [supported languages]({{< ref "api-management/plugins/overview#plugin-driver-names" >}}) section for list of supported plugin driver names. + +Each plugin can have additional settings, such as: +- `disabled`: When true, disables the plugin. +- `raw_body_only`: When true, indicates that only the raw body should be processed. +- `require_session`: When true, indicates that session metadata will be available to the plugin. This is applicable only for post, post authentication and response plugins. + +#### Using API Designer {#tyk-classic-dashboard} + +This section explains how to configure plugins for a Tyk Classic API using Tyk Dashboard. It specifically covers the use case where the source files of your plugins are deployed on the Tyk Gateway file system. + +Select your API from the list of *Created APIs* to reach the API designer and then follow these steps: + +{{< img src="/img/plugins/plugins_classic_api_source_config.png" alt="Plugins Classic API screen" >}} + +1. **Display the Tyk Classic API Definition editor** + + Click on the **View Raw Definition** button to display an editor for updating the Tyk Classic API Definition. + + {{< img src="/img/plugins/plugins_classic_api_definition_editor.png" alt="Plugins Classic API Definition editor screen" >}} + +2. **Edit the Tyk Classic API Definition to configure plugins** + + Use the editor to edit the `custom_middleware` section of the [Tyk Classic API Definition]({{< ref "api-management/plugins/overview#tyk-classic-apis" >}}). + + {{< img src="/img/plugins/plugins_classic_api_bundles_config.png" alt="Plugins Classic API Bundle Field" >}} + +3. **Save changes** + + Select the **Update** button to apply your changes to the Tyk Classic API Definition. + +## Plugin Deployment Types + +There are a variety of scenarios relating to the deployment of plugins for an API, concerning the location of the plugin source code and its associated configuration. + +### Local Plugins + +The plugin source code and associated configuration are co-located with Tyk Gateway in the same file system. The configuration is located within the API Definition. For further details please consult [API configuration]({{< ref "api-management/plugins/overview#api-configuration" >}}). + +### Plugin Bundles (Remote) + +The plugin source code and associated configuration are bundled into a zip file and uploaded to a remote webserver. Multiple plugins can be stored in a single *plugin bundle*. Tyk Gateway will download the plugin bundle from the remote webserver and then extract, cache and execute plugins for each of the configured phases of the API request / response lifecycle. For further details on plugin bundles and how to configure them, please refer to the [plugin bundles]({{< ref "api-management/plugins/overview#plugin-bundles" >}}) page. + +### gRPC Plugins (Remote) + +Custom plugins can be hosted on a remote server and executed from the Tyk Gateway middleware chain via gRPC. These plugins can be written in any language you prefer, as they are executed on the gRPC server. You'll configure your API definition so that Tyk Gateway will send requests to your gRPC server at the appropriate points in the API request / response lifecycle. For further details please consult our [gRPC]({{< ref "api-management/plugins/rich-plugins#overview-1" >}}) documentation. + +## Plugin Bundles + +For Tyk Gateway to execute local custom plugins during the processing of API requests and responses, the plugin source code must be loaded into the Gateway. The source is usually stored in files and the API definition is used to point the Gateway at the correct file for each [plugin type]({{< ref "api-management/plugins/plugin-types#plugin-types" >}}). To simplify the management of plugins, you can group (or *bundle*) multiple plugin files together in a ZIP file that is referred to as a *plugin bundle*. + +### When To Use Plugin Bundles + +Plugin bundles are intended to simplify the process of attaching and loading custom middleware. Multiple API definitions can refer to the same plugin bundle (containing the source code and configuration) if required. Having this common, shared resource avoids you from having to duplicate plugin configuration for each of your APIs definitions. + +### How Plugin Bundles Work + +The source code and a [manifest file](#manifest) are bundled into a zip file and uploaded to an external remote web server. The manifest file references the source code file path and the function name within the code that should be invoked for each [plugin type]({{< ref "api-management/plugins/plugin-types#plugin-types" >}}). Within the API definition, custom plugins are configured simply using the name of the bundle (zip file). Tyk Gateway downloads, caches, extracts and executes plugins from the downloaded bundle according to the configuration in the manifest file. + +{{< img src= "/img/plugins/plugin-bundles-overview.png" alt="plugin bundles architectural overview" >}} + +#### Caching plugin bundles + +Tyk downloads a plugin bundle on startup based on the configuration in the API definition, e.g. `http://my-bundle-server.com/bundles/bundle-latest.zip`. The bundle contents will be cached so that, when a Tyk reload event occurs, the Gateway does not have to retrieve the bundle from the server again each time. If you want to use a different bundle then you must update your API to retrieve a different bundle filename and then trigger a reload. It is not sufficient simply to replace the bundle file on your server with an updated version with the same name - the caching ensures this will not be retrieved during a reload event. + +As a suggestion, you may organize your plugin bundle files using a Git commit reference or version number, e.g. `bundle-e5e6044.zip`, `bundle-48714c8.zip`, `bundle-1.0.0.zip`, `bundle-1.0.1.zip`, etc. + +Alternatively, you may delete the cached bundle from Tyk manually and then trigger a hot reload to tell Tyk to fetch a new one. By default, Tyk will store downloaded bundles in this path: +`{ TYK_ROOT } / { CONFIG_MIDDLEWARE_PATH } / bundles` + +#### Gateway configuration + +To configure Tyk Gateway to load plugin bundles the following parameters must be specified in your `tyk.conf`: + +```yaml +"enable_bundle_downloader": true, +"bundle_base_url": "http://my-bundle-server.com/bundles/", +"public_key_path": "/path/to/my/pubkey", +``` + +- `enable_bundle_downloader`: Enables the bundle downloader. +- `bundle_base_url`: A base URL that will be used to download the bundle. For example if we have `bundle-latest.zip` specified in the API definition, Tyk will fetch the following file: `http://my-bundle-server.com/bundles/bundle-latest.zip` (see the next section for details). +- `public_key_path`: Sets a public key, used for verifying signed bundles. If unsigned bundles are used you may omit this. + +{{< note success >}} +**Note** + +Remember to set `"enable_coprocess": true` in your `tyk.conf` when using [rich plugins]({{< ref "api-management/plugins/overview#plugin-bundles" >}})! +{{< /note >}} + +#### The manifest file {#manifest} + +A plugin bundle must include a manifest file (called `manifest.json`). The manifest file contains important information like the configuration block and the list of source code files that will be included as part of the bundle file. If a file isn't specified in the list, it won't be included in the resulting file, even if it's present in the current directory. + +A sample manifest file looks like this: + +```json +{ + "file_list": [ + "middleware.py", + "mylib.py" + ], + "custom_middleware": { + "pre": [ + { + "name": "PreMiddleware" + } + ], + "post": [ + { + "name": "PostMiddleware" + } + ], + "driver": "python" + }, + "checksum": "", + "signature": "" +} +``` + +You may leave the `checksum` and `signature` fields empty, the bundler tool will fill these during the build process. + +The `custom_middleware` block follows the standard syntax we use for Tyk plugins. In Tyk Community Edition, where file-based API configuration is used by default, a `custom_middleware` block is located/added to the API configuration file. + +#### Creating plugin bundles + +Tyk provides the Bundle CLI tool as part of the `tyk` binary. For further details please visit the [Bundle CLI tool]({{< ref "api-management/plugins/overview#bundler-cli-tool" >}}) page. + +### Tyk OAS API Configuration + +For API plugins that are deployed as [plugin bundles]({{< ref "api-management/plugins/overview#plugin-bundles" >}}), the API should be configured with the name of the plugin bundle file to download from your remote web server. Furthermore, the Gateway should be [configured]({{< ref "api-management/plugins/overview#gateway-configuration" >}}) to enable downloading plugin bundles. + +You can configure your API with the name of the plugin bundle file to download within the Tyk OAS API definition or API Designer. + +If you’re using the legacy Tyk Classic APIs, then check out the [Tyk Classic]({{< ref "api-management/plugins/overview#tyk-classic-apis" >}}) page. + +#### Using API Definition + +The configuration for a Tyk OAS API to fetch the download of a plugin bundle from a remote web server is encapsulated within the `pluginConfig` section within the `middleware.global` section of the `x-tyk-api-gateway` part of a Tyk OAS API Definition. + +The `pluginConfig` section is structured as follows: + +- `bundle`: A JSON entity that contains the following configuration parameters: + - `enabled`: When `true`, enables the plugin. + - `path`: The relative path of the zip file in relation to the base URL configured on the remote webserver that hosts plugin bundles. +- `driver`: Indicates the type of plugin, e.g. `golang`, `grpc`, `lua`, `otto` or `python`. + +An illustrative example is listed below: + +```json{hl_lines=["37-45"], linenos=true, linenostart=1} +{ + "components": {}, + "info": { + "title": "example-oas-plugin-configuration", + "version": "1.0.0" + }, + "openapi": "3.0.3", + "paths": { + "/anything": { + "put": { + "operationId": "anythingput", + "responses": { + "200": { + "description": "" + } + } + } + } + }, + "x-tyk-api-gateway": { + "info": { + "name": "example-oas-plugin-configuration", + "state": { + "active": true + } + }, + "upstream": { + "url": "http://httpbin.org/" + }, + "server": { + "listenPath": { + "value": "/example-oas-plugin-configuration/", + "strip": true + } + }, + "middleware": { + "global": { + "pluginConfig": { + "bundle": { + "enabled": true, + "path": "plugin.zip" + }, + "driver": "goplugin" + } + } + } + } +} +``` + +In this example we can see that bundle plugin has been configured within the `middleware.global.pluginConfig.bundle` object. The plugin is enabled and bundled within file `plugin.zip`. The plugin bundle is a Go plugin, i.e. `middleware.global.pluginConfig.driver` has been configured with value `goplugin`. + +The configuration above is a complete and valid Tyk OAS API Definition that you can import into Tyk to try out custom plugin bundles, assuming that you have provided a valid bundle file named `plugin.zip`. + +#### Using API Designer + +To configure plugin bundles for Tyk OAS APIs click on the APIs menu item in the *API Management* menu of Dashboard and select your API to display the editor screen. Subsequently, follow the steps below: + +1. **Access plugin options** + + Scroll down until the *Enable Plugin* section is displayed. + + {{< img src="/img/plugins/plugins_oas_api_bundles_config.png" alt="Tyk OAS API Bundle section" >}} + +2. **Enable plugin bundle for you API** + + Enable a plugin bundle for your API by activating the toggle switch. + +3. **Enter relative path to plugin bundle file** + + Enter the relative path of the plugin bundle file in the *Plugin Bundle ID* field that Tyk Gateway should download from the web server that hosts your plugin bundles. + +4. **Save the API** + + Select **Save API** to apply the changes to your API. + +### Tyk Classic API Configuration + +For custom plugins that are deployed as [plugin bundles]({{< ref "api-management/plugins/overview#plugin-bundles" >}}), the API should be configured with the name of the plugin bundle file to download from your remote web server. Furthermore, the Gateway should be [configured]({{< ref "api-management/plugins/overview#gateway-configuration" >}}) to enable downloading plugin bundles. + +You can configure your API with the name of the plugin bundle file to download within the Tyk Classic API definition or API Designer. + +If you’re using the newer Tyk OAS APIs, then check out the [Tyk OAS]({{< ref "api-management/plugins/overview#tyk-oas-api-configuration" >}}) page. + +#### Using API Definition + +The configuration for an API to fetch and download a plugin bundle from a remote server is encapsulated within the `custom_middleware_bundle` field of the Tyk Classic API Definition. An illustrative example is listed below: + +```json {hl_lines=["33"], linenos=true, linenostart=1} +{ + "name": "Tyk Classic Bundle API", + "api_id": "1", + "org_id": "default", + "definition": { + "location": "header", + "key": "version" + }, + "auth": { + "auth_header_name": "authorization" + }, + "use_keyless": true, + "version_data": { + "not_versioned": true, + "versions": { + "Default": { + "name": "Default", + "expires": "3000-01-02 15:04", + "use_extended_paths": true, + "extended_paths": { + "ignored": [], + "white_list": [], + "black_list": [] + } + } + } + }, + "proxy": { + "listen_path": "/quickstart/", + "target_url": "http://httpbin.org", + "strip_listen_path": true + }, + "custom_middleware_bundle": "bundle-latest.zip" +} +``` + +With the configuration given in the example above, calls to the API will invoke the custom plugins defined in the `manifest.json` file contained within `bundle-latest.zip` uploaded to your remote webserver, e.g. `http://your-example-plugin-server.com/plugins`. + +Tyk Gateway should be configured for downloading plugin bundles from a secured web server. Please consult the [plugin bundles]({{< ref "api-management/plugins/overview#plugin-bundles" >}}) documentation for further details. + +#### Using API Designer + +To configure plugin bundles for Tyk Classic APIs click on the APIs menu item in the *API Management* menu of Dashboard and select your API to display the API editor screen. Subsequently, follow the steps below: + +1. **Access plugin options** + + Click on the *Advanced Options* tab and scroll down until the *Plugin Options* section is displayed. + + {{< img src="/img/plugins/plugins_classic_api_bundles_config.png" alt="Tyk Classic Plugin Options section" >}} + +2. **Enter relative path to bundle file** + + Enter the relative path of the plugin bundle file in the *Plugin Bundle ID* field that Tyk Gateway should download from the web server hosting plugin bundles. + +3. **Save the API** + + Select the **save** or **update** button to apply the changes to your API. + +### Bundler CLI Tool + +The bundler tool is a CLI service, provided by _Tyk Gateway_ as part of its binary since v2.8. This lets you generate +[plugin bundles]({{< ref "api-management/plugins/overview#plugin-bundles" >}}). + +{{< note >}} +**Note** +Generated plugin bundles must be served using your own web server. +{{< /note >}} + +Issue the following command to see more details on the `bundle` command: + +```bash +/opt/tyk-gateway/bin/tyk bundle -h +``` + +--- + +#### Prerequisites + +To create plugin bundles you will need the following: + +- **Manifest.json**: The [manifest.json]({{< ref "api-management/plugins/overview#manifest" >}}) file + contains the paths to the plugin source files and the name of the function implementing each plugin. The + _manifest.json_ file is mandatory and must exist on the Tyk Gateway file system. By default the bundle CLI looks for + a file named _manifest.json_ in the current working directory where the bundle command is run from. The exact location + can be specified using the `--manifest` command option. +- **Plugin source code files**: The plugin source code files should be contained relative to the directory in which the + _manifest.json_ file is located. The _manifest.json_ should contain relative path references to source code files. + + {{< note >}} + **Note** + Source code files are not required when creating a plugin bundle for gRPC plugins since the plugin + source code is located at the gRPC server. + {{< /note >}} + +- **Certificate key**: Plugin bundles can optionally be signed with an RSA private key. The corresponding public key + should be located in the file configured in environmental variable `TYK_GW_PUBLICKEYPATH` or the `public_key_path` + parameter in `tyk.conf`: + +```json +{ + "enable_bundle_downloader": true, + "bundle_base_url": "http://my-bundle-server.com/bundles/", + "public_key_path": "/path/to/my/pubkey.pem" +} +``` + +--- + +#### Directory Structure + +A suggested directory structure is shown below for Golang, Javascript and Python bundles in the tabs below. + +{{< note success >}} +**Note** + +Sub-directories (folders) are not supported inside the `bundle-directory` location. + +{{< /note >}} + +{{< tabs_start >}} {{< tab_start "Golang" >}} + +```bash +/bundle-directory +├── manifest.json # Manifest file with plugin references +└── plugin.so # Compiled Golang plugin +``` + +{{< tab_end >}} {{< tab_start "Javascript" >}} + +```bash +/bundle-directory +├── manifest.json # Manifest file with plugin references +├── plugin1.js # First JavaScript plugin source file +└── plugin2.js # Second JavaScript plugin source file +``` + +{{< tab_end >}} + +{{< tab_start "Python" >}} + +```bash +/bundle-directory +├── manifest.json # Manifest file with plugin references +├── plugin1.py # First Python plugin source file +└── plugin2.py # Second Python plugin source file +``` + +{{< tab_end >}} + +{{< tabs_end >}} + +The `manifest.json` will reference the files located in the `bundle-directory`, ensure plugin source files are organized relative to the manifest. The Tyk Gateway will load and execute these plugins based on the paths defined in the `manifest.json` file. + +Sample `manifest.json` is shown below for Golang, Javascript and Python bundles in the tabs below. + +{{< tabs_start >}} {{< tab_start "Golang" >}} + +```json +{ + "file_list": [ + "plugin.so" + ], + "custom_middleware": { + "pre": [ + { + "name": "PreMiddleware", + "path": "./plugin.so" + } + ], + "post": [ + { + "name": "PostMiddleware", + "path": "./plugin.so" + } + ], + "driver": "goplugin" + }, + "checksum": "", + "signature": "" +} + +``` + +{{< tab_end >}} {{< tab_start "Javascript" >}} + +```json +{ + "file_list": [ + "plugin1.js", + "plugin2.js" + ], + "custom_middleware": { + "pre": [ + { + "name": "PreMiddleware", + "path": "./plugin1.js" + } + ], + "post": [ + { + "name": "PostMiddleware", + "path": "./plugin2.js" + } + ], + "driver": "otto" + }, + "checksum": "", + "signature": "" +} +``` + +{{< tab_end >}} + +{{< tab_start "Python" >}} + +```json +{ + "file_list": [ + "plugin1.py", + "plugin2.py" + ], + "custom_middleware": { + "pre": [ + { + "name": "PreMiddleware", + "path": "./plugin1.py" + } + ], + "post": [ + { + "name": "PostMiddleware", + "path": "./plugin2.py" + } + ], + "driver": "python" + }, + "checksum": "", + "signature": "" +} +``` + +{{< tab_end >}} + +{{< tabs_end >}} + +--- + +#### Creating a plugin bundle + +Run the following command to create the bundle: + +```bash +$ tyk bundle build +``` + +The resulting file will contain all your specified files and a modified `manifest.json` with the checksum and signature +(if required) applied, in ZIP format. + +By default, Tyk will attempt to sign plugin bundles for improved security. If no private key is specified, the program +will prompt for a confirmation. Use `-y` to override this (see options below). + +--- + +#### Command Options + +Instructions on how to create plugin bundles is displayed by issuing the following command: + +```bash +/opt/tyk-gateway/bin/tyk bundle build -h +``` + +The following options are supported: + +- `--manifest`: Specifies the path to the manifest file. This defaults to `manifest.json` within the current working + directory. +- `--output`: Specifies the name of the bundle file e.g. `--output bundle-latest.zip`. If this flag is not specified, + `bundle.zip` will be used. +- `-y`: Force tool to create unsigned bundle without prompting e.g. `$ tyk bundle build --output bundle-latest.zip -y`. +- `--key`: Specifies the path to your private key which is used to generate signed bundle e.g. + `$ tyk bundle build --output bundle-latest.zip --key=mykey.pem`. + +--- + +#### Docker Example + +Since v5.5 Tyk Gateway uses distroless docker images. + +For Gateway version < v5.5 it is possible to use Docker to create plugin bundles as shown in the example below. + +```bash +docker run --rm -it \ + --name bundler \ + -v `pwd`:/plugin-source \ + -v `pwd`/../../../confs/keys:/keys \ + -w /plugin-source \ + --entrypoint /bin/bash \ + tykio/tyk-gateway:v5.4.0 \ + -c 'export PATH="/opt/tyk-gateway:$$PATH"; tyk bundle build -o bundle.zip -k /keys/key.pem' +``` + +This Docker command runs a container using the `tykio/tyk-gateway:v5.4.0` image to build a Tyk plugin bundle. It mounts +the current directory from the host as `/plugin-source` and a directory containing keys as `/keys` inside the container. +The working directory within the container is set to `/plugin-source`, and the default entrypoint is overridden to use +`/bin/bash`. The command executed in the container exports a modified `PATH` to include the Tyk Gateway binaries, then +runs `tyk bundle build` to generate a plugin bundle named `bundle.zip`, using the specified key for authentication. The +container is automatically removed after the command completes, and the operation is conducted interactively. + +## Supported Languages + +The following languages are supported for custom plugins: +* [Golang]({{< ref "api-management/plugins/golang#" >}}): A plugin written in Golang is called a **Native Plugin**. Tyk recommends using Go plugins for performance, flexibility, and nativity reasons (all Tyk components are written in Go). +* [JavaScript]({{< ref "api-management/plugins/javascript#" >}}): A plugin written in Javascript uses JavaScript Virtual Machine (JSVM) interpreter. +* [Rich Plugins]({{< ref "api-management/plugins/rich-plugins#" >}}) includes Python, Lua, gRPC - With gRPC, you can write plugins in Java, .NET, C++ / C#, PHP, and all other [gRPC supported languages](https://grpc.io/docs/languages/). +Rich plugins give ultimate flexibility in the language of implementation, however, there are some performance and management overheads when compared with native GoLang plugin. + +**Common To All Plugin Languages:** + +* Make Layer 4 (TCP) or Layer 7 (HTTP/REST/SOAP) calls +* Open Persistent Connections +* Modify the request in-flight +* Used to stop the request and return a [custom response]({{< ref "api-management/plugins/plugin-types#return-overrides--returnoverrides" >}}) +* Be served using [Bundles]({{< ref "api-management/plugins/overview#plugin-deployment-types" >}}) or by files on the file system, except gRPC of course which by definition is served by some webserver in the language of your choosing + +### Plugin Hook Types + +Tyk provide 5 different phases, i.e. hooks to inject custom plugin throughout the [API execution lifecycle]({{< ref "concepts/middleware-execution-order" >}}). + +Not all hooks are supported in every language. The following table shows you which plugin language support which phase/hook: + +| | Auth | Pre | Post-Auth | Post | Response +|------------|--------|----------|-----------|------|-----------| +| GoLang | ✅ |✅ |✅ |✅ |✅ +| JavaScript | ❌ |✅ |❌ |✅ |❌ +| gRPC | ✅ |✅ |✅ |✅ |✅ +| Python | ✅ |✅ |✅ |✅ |✅ +| Lua | ✅ |✅ |✅ |✅ |❌ + +More reading on the [hook types]({{< ref "api-management/plugins/rich-plugins#coprocess-dispatcher---hooks" >}}) in rich plugins and explanation with common use case for each [hook type]({{}}) + + +### Plugin Driver Names + +We use the following Plugin driver names: + +| Plugin | Name | +| ---------- | --------- | +| GoLang | goplugin | +| JavaScript | otto | +| gRPC | grpc | +| Python | python | +| Lua | lua | + +### Limitations + +What are the limitations to using this programming Language? + +| | GoLang | JavaScript | gRPC | Python | Lua +|---|--------|------------------|-----------|-----------|-----------| +| Runs in Gateway process | ✅
Runs
natively | ✅
Built-In JSVM Interpreter | ❌
Standalone server | ✅
Tyk talks with Python interpreter |✅ +| Built-in SDK | ✅
All Gateway Functionality | ✅
[Yes]({{< ref "api-management/plugins/javascript#javascript-api" >}}) | ❌ | ✅
[Yes]({{< ref "api-management/plugins/rich-plugins#tyk-python-api-methods" >}}) | ❌ +| TCP Connections

(DBs, Redis, etc)

| ✅ | ❌
Very Limited | ✅ | ✅ | ✅ | + +### Custom Plugin Table + +We have put together a [GitHub repo with a table of custom plugins](https://github.com/TykTechnologies/custom-plugins#custom-gateway-plugins) in various languages that you can experiment with. If you would like to submit one that you have developed, feel free to open an issue in the repo. + +### Differences between Rich Plugins and JSVM middleware + +#### JavaScript +The JavaScript Virtual Machine provides pluggable middleware that can modify a request on the fly and are designed to augment a running Tyk process, are easy to implement and run inside the Tyk process in a sandboxed *ECMAScript* interpreter. This is good, but there are some drawbacks with the JSVM: + +* **Performance**: JSVM is performant, but is not easy to optimize and is dependent on the [otto interpreter](https://github.com/robertkrimen/otto) - this is not ideal. The JSVM also requires a copy of the interpreter object for each request to be made, which can increase memory footprint. +* **Extensibility**: JSVM is a limited interpreter, although it can use some NPM modules, it isn't NodeJS so writing interoperable code (especially with other DBs) is difficult. +* **TCP Access**: The JSVM has no socket access so working with DB drivers and directly with Redis is not possible. + +#### Rich Plugins +Rich Plugins can provide replacements for existing middleware functions (as opposed to augmentation) and are designed to be full-blown, optimized, highly capable services. They enable a full customized architecture to be built that integrates with a user's infrastructure. + +Rich Plugins bring about the following improvements: + +* **Performance**: Run on STDIN (unix pipes), which are extremely fast and run in their own memory space, and so can be optimized for performance way beyond what the JSVM could offer. +* **Extensibility**: By allowing any language to be used so long as GRPC is supported, the extensibility of a CPH is completely open. +* **TCP Access**: Because a plugin is a separate process, it can have it's own low-level TCP connections opens to databases and services. + +## Plugin Caveats + +- Tyk Gateway manages plugins for each API within the same process. +- For [gRPC plugins]({{< ref "api-management/plugins/rich-plugins#overview-1" >}}), Tyk Gateway can only be configured to integrate with one gRPC server. +- Javascript plugins only allow Pre and Post Request hooks of the API Request Lifecycle. + + +## Plugins Hub + + + +Welcome to the Tyk Plugins Hub, dedicated to providing you with a curated list of resources that showcase how to develop Tyk Plugins. + +[Tyk Plugins]({{< ref "api-management/plugins/overview#" >}}) are a powerful tool that allows you to develop custom middleware that can intercept requests at different stages of the request lifecycle, modifying/transforming headers and body content. + +Tyk has extensive support for writing custom plugins using a wide range of languages, most notably: Go, Python, Javascript etc. In fact, plugins can be developed using most languages via *gRPC*. + +### Blogs + +Selected blogs for plugin development are included below. Further examples are available at the Tyk [website](https://tyk.io/?s=plugin). + +1. **[Decoupling micro-services using Message-based RPC](https://medium.com/@asoorm/decoupling-micro-services-using-message-based-rpc-fa1c12409d8f)** + + - **Summary**: Explains how to write a plugin that intercepts an API request and forwards it to a gRPC server. The gRPC server processes the request and dispatches work to an RabbitMQ message queue. The source code is available in the accompanying [GitHub repository](https://github.com/asoorm/tyk-rmq-middleware) + +2. **[How to configure a gRPC server using Tyk](https://tyk.io/blog/how-to-configure-a-grpc-server-using-tyk/)** + + - **Summary**: Explains how to configure a Python implementation of a gRPC server to add additional logic to API requests. During the request lifecycle, the Tyk-Gateway acts as a gRPC client that contacts the Python gRPC server, providing additional custom logic. + +3. **[How to deploy Python plugins in Tyk running On Kubernetes](https://tyk.io/blog/how-to-deploy-python-plugins-in-tyk-running-on-kubernetes/)** + + - **Summary**: Explains how to deploy a custom Python plugin into a Tyk installation running on a Kubernetes cluster. + +### GitHub Repositories + +Here are some carefully selected GitHub repositories that will help you learn how to integrate and utilize Tyk Plugins in your development projects: + +1. **[Tyk Awesome Plugins](https://github.com/TykTechnologies/tyk-awesome-plugins)** + + - **Description**: Index of plugins developed using a variety of languages. + - **Key Features Demonstrated**: A comprehensive index for a collection of plugins that can be used with the Tyk API Gateway in areas such as: rate limiting, authentication and request transformation. The examples are developed using a diverse array of languages, including but not limited to: Python, JavaScript and Go. This broad language support ensures that developers from different backgrounds and with various language preferences can seamlessly integrate these plugins with their Tyk API Gateway implementations. + +2. **[Custom Plugin Examples](https://github.com/TykTechnologies/custom-plugin-examples/tree/master)** + + - **Description**: Index of examples for a range of plugin hooks (Pre, Post, Post-Auth and Response) developed using a variety of languages. + - **Key Features Demonstrated**: Specific examples include invoking an AWS lambda, inserting a new claim into a JWT, inject a signed JWT into authorization header, request header modification. A range of examples are available including Python, Java, Ruby, Javascript, NodeJS and Go. + +3. **[Environment For Plugin Development](https://github.com/TykTechnologies/custom-go-plugin)** + + - **Description**: Provides a docker-compose environment for developing your own custom Go plugins. + - **Key Features Demonstrated**: Showcases support for bundling plugins, uploading plugins to AWS S3 storage, test coverage etc. + + diff --git a/tyk-docs/content/api-management/plugins/plugin-types.md b/tyk-docs/content/api-management/plugins/plugin-types.md new file mode 100644 index 0000000000..43b4429711 --- /dev/null +++ b/tyk-docs/content/api-management/plugins/plugin-types.md @@ -0,0 +1,743 @@ +--- +title: "Plugin Types" +date: 2025-01-10 +tags: ["Dashboard", "User Management", "RBAC", "Role Based Access Control", "User Groups", "Teams", "Permissions", "API Ownership", "SSO", "Single Sing On", "Multi Tenancy"] +description: "How to manage users, teams, permissions, rbac in Tyk Dashboard" +keywords: ["Dashboard", "User Management", "RBAC", "Role Based Access Control", "User Groups", "Teams", "Permissions", "API Ownership", "SSO", "Single Sing On", "Multi Tenancy"] +aliases: + - /plugins/plugin-types/plugintypes + - /plugins/plugin-types/request-plugins + - /plugins/plugin-types/auth-plugins/auth-plugins + - /plugins/plugin-types/auth-plugins/id-extractor + - /plugins/plugin-types/response-plugins + - /plugins/plugin-types/analytics-plugins + - /product-stack/tyk-gateway/middleware/endpoint-plugin + - /product-stack/tyk-gateway/middleware/endpoint-plugin-tyk-oas + - /product-stack/tyk-gateway/middleware/endpoint-plugin-tyk-classic + - /plugins/request-plugins + - /plugins/auth-plugins + - /customise-tyk/plugins/rich-plugins/id-extractor + - /plugins/rich-plugins/id-extractor + - /plugins/auth-plugins/id-extractor + - /plugins/response-plugins + - /plugins/analytics-plugins +--- + +## Introduction + +Custom Plugins enable users to execute custom code to complete tasks specific to their use case, allowing users to complete tasks that would not otherwise be possible using Tyk’s standard middleware options. + +Tyk has a [pre-defined execution order]({{< ref "/concepts/middleware-execution-order" >}}) for the middleware which also includes **seven hooks** for the custom plugins. As such, users can execute, or `hook`, their plugin in these phases of the API request/response lifecycle based on their specific use case. + +## Plugin and Hook Types +This table includes all the plugin types with the relevant hooks, their place in the execution chain, description and examples: + +| Hook Type (in their execution order) | Plugin Type | HTTP Request/Response phase | Executed before/after reverse proxy to the upstream API | Details | Common Use Cases | +|--------------------------|----|---|--------------|--------------------|--------- +| Pre (Request) | Request Plugin | HTTP request | Before | The first thing to be executed, before any middleware | IP Rate Limit plugins, API Request enrichment | +| Authentication| Authentication Plugin | HTTP request | Before | Replaces Tyk's authentication & authorization middleware with your own business logic | When you need your a custom flow, for example, interfacing with legacy Auth database | +| Post-Auth (Request)| Authentication Plugin | HTTP request | Before | Executed immediately after authentication middleware | Additional special custom authentication is needed | +| Post (Request)| Request Plugin | HTTP request| Before | The final middleware to be executed during the *HTTP request* phase (see **Note** below) | Update the request before it gets to the upstream, for example, adding a header that might override another header, so we add it at the end to ensure it doesn't get overridden | +| Response Plugin| Response Plugin | HTTP Response | After | Executed after the reverse proxy to the upstream API | Executed straight after the reverse proxy returns from the upstream API to Tyk | Change the response before the user gets it, for example, change `Location` header from internal to an external URL | +| Analytics Plugin (Request+Response)| Analytics Plugin | HTTP request | After | The final middleware to be executed during the *HTTP response* phase | Change analytics records, for example, obfuscating sensitive data such as the `Authorization` header | + +{{< note success >}} +**Note** + +There are two different options for the Post Plugin that is executed at the end of the request processing chain. The API-level Post Plugin is applied to all requests, whilst the [endpoint-level]({{< ref "api-management/plugins/plugin-types#per-endpoint-custom-plugins" >}}) custom Golang plugin is only applied to requests made to specific endpoints. If both are configured, the endpoint-level plugin will be executed first. +{{< /note >}} + +## Plugin Types + +Tyk supports four types of plugins: + +1. **[Request Plugin]({{< ref "" >}})** +2. **[Authentication Plugin]({{< ref "" >}})** +3. **[Response Plugin]({{< ref "" >}})** +4. **[Analytics Plugin]({{< ref "" >}})** + +## Request Plugins + +There are 4 different phases in the [request lifecycle]({{< ref "concepts/middleware-execution-order" >}}) you can inject custom plugins, including [Authentication plugins]({{< ref "api-management/plugins/plugin-types#authentication-plugins" >}}). There are performance advantages to picking the correct phase, and of course that depends on your use case and what functionality you need. + +### Hook Capabilities +| Functionality | Pre | Auth | Post-Auth | Post | +|-------------------------|----------|-------------|-----------|-----------| +| Can modify the Header | ✅ | ✅ | ✅ | ✅ +| Can modify the Body | ✅ | ✅ | ✅ |✅ +| Can modify Query Params | ✅ | ✅ | ✅ |✅ +| Can view Session1 Details (metadata, quota, context-vars, tags, etc) | ❌ | ✅ |✅ |✅ +| Can modify Session1 2 | ❌ | ✅ | ❌ |❌ +| Can Add More Than One3 | ✅ | ❌ |✅ | ✅ + +1. A [Session object]({{< ref "getting-started/key-concepts/what-is-a-session-object" >}}) contains allowances and identity information that is unique to each requestor + +2. You can modify the session by using your programming language's SDK for Redis. Here is an [example](https://github.com/TykTechnologies/custom-plugins/blob/master/plugins/go-auth-multiple_hook_example/main.go#L135) of doing that in Golang. + +3. For select hook locations, you can add more than one plugin. For example, in the same API request, you can have 3 Pre, 1 auth, 5 post-auth, and 2 post plugins. + +### Return Overrides / ReturnOverrides +You can have your plugin finish the request lifecycle and return a response with custom payload & headers to the requestor. + +[Read more here]({{< ref "api-management/plugins/rich-plugins#returnoverrides" >}}) + +##### Python Example + +```python +from tyk.decorators import * + +@Hook +def MyCustomMiddleware(request, session, spec): + print("my_middleware: MyCustomMiddleware") + request.object.return_overrides.headers['content-type'] = 'application/json' + request.object.return_overrides.response_code = 200 + request.object.return_overrides.response_error = "{\"key\": \"value\"}\n" + return request, session +``` + +##### JavaScript Example +```javascript +var testJSVMData = new TykJS.TykMiddleware.NewMiddleware({}); + +testJSVMData.NewProcessRequest(function(request, session, config) { + request.ReturnOverrides.ResponseError = "Foobarbaz" + request.ReturnOverrides.ResponseBody = "Foobar" + request.ReturnOverrides.ResponseCode = 200 + request.ReturnOverrides.ResponseHeaders = { + "X-Foo": "Bar", + "X-Baz": "Qux" + } + return testJSVMData.ReturnData(request, {}); +}); +``` + + +## Authentication Plugins + +If you have unique authentication requirements, you can write a custom authentication plugin. + +### Session Authentication and Authorization + +A very important thing to understand when using custom authentication plugins is that Tyk will continue to perform session authentication and authorization using the information returned by your plugin. Tyk will cache this Session information. **This is necessary in order to do things like rate limiting, access control, quotas, throttling, etc.** + +Tyk will try to be clever about what to cache, but we need to help it. There are two ways to do that, with and without the `ID Extractor`. + +#### The ID Extractor + +The ID Extractor is a caching mechanism that's used in combination with Tyk Plugins. It can be used specifically with plugins that implement custom authentication mechanisms. The ID Extractor works for all rich plugins: gRPC-based plugins, Python and Lua. + +See [ID Extractor]({{< ref "api-management/plugins/plugin-types#plugin-caching-mechanism" >}}) for more details. + +#### Token Metadata + +Tyk creates an in-memory object to track the rate limit, quotas, and more for each session. + +This is why we set the `token` metadata when using custom authentication middleware, in order to give Tyk a unique ID with which to track each session. + +For backwards compatibility, even when using an ID Extractor, we need to continue to set the `token` metadata. For example, when building a session object in GoLang custom middleware: + +```{.copyWrapper} +object.Session = &coprocess.SessionState{ + LastUpdated: time.Now().String(), + Rate: 5, + Per: 10, + QuotaMax: int64(0), + QuotaRenews: time.Now().Unix(), + IdExtractorDeadline: extractorDeadline, + Metadata: map[string]string{ + "token": "my-unique-token", + }, + ApplyPolicies: ["5d8929d8f56e1a138f628269"], + } +``` +[source](https://github.com/TykTechnologies/tyk-grpc-go-basicauth-jwt/blob/master/main.go#L102) + +#### Without ID Extractor + +When not using ID Extractor, Tyk will continue to cache authenticated sessions returned by custom auth plugins. We must set a unique `token` field in the Metadata (see above) that Tyk will use to cache. + +### Supported Languages + +The following languages are supported for custom authentication plugins: + +- All Rich Plugins (gRPC, Python, Lua) +- GoLang + +See the [supported languages]({{< ref "api-management/plugins/overview#supported-languages" >}}) section for custom authentication plugin examples in a language of your choosing. There's also a [blog that walks you through setting up gRPC custom auth in Java](https://tyk.io/how-to-setup-custom-authentication-middleware-using-grpc-and-java/). + +### Tyk Operator + +Please consult the Tyk Operator supporting documentation for examples of how to configure a Tyk Operator API to use: + +- [Go custom authentication plugin]({{< ref "api-management/automations/operator#custom-plugin-auth-go" >}}) +- [gRPC custom authentication plugin]({{< ref "api-management/automations/operator#custom-plugin-auth-grpc" >}}) + +## Response Plugins + +Since Tyk 3.0 we have incorporated response hooks, this type of hook allows you to modify the response object returned by the upstream. The flow is follows: + +- Tyk receives the request. +- Tyk runs the full middleware chain, including any other plugins hooks like Pre, Post, Custom Authentication, etc. +- Tyk sends the request to your upstream API. +- The request is received by Tyk and the response hook is triggered. +- Your plugin modifies the response and sends it back to Tyk. +- Tyk takes the modified response and is received by the client. + +This snippet illustrates the hook function signature: + +```python +@Hook +def ResponseHook(request, response, session, metadata, spec): + tyk.log("ResponseHook is called", "info") + # In this hook we have access to the response object, to inspect it, uncomment the following line: + # print(response) + tyk.log("ResponseHook: upstream returned {0}".format(response.status_code), "info") + # Attach a new response header: + response.headers["injectedkey"] = "injectedvalue" + return response +``` + +If working with a Tyk Classic API, you would add this configuration to the API definition: + +``` +{ + "custom_middleware": { + "response": [ + { + "name": "ResponseHook", + "path": "middleware/middleware.py" + } + ], + "driver": "python" + } +} +``` + + - `driver`: set this to the appropriate value for the plugin type (e.g. `python`, `goplugin`) + - `response`: this is the hook name. You use middleware with the `response` hook type because you want this custom middleware to process the request on its return leg of a round trip. + - `response.name`: is your function name from the plugin file. + - `response.path`: is the full or relative (to the Tyk binary) path to the plugin source file. Ensure Tyk has read access to this file. + +Starting from versions 5.0.4 and 5.1.1+ for our Go, Python and Ruby users we have introduced the `multivalue_headers` field to facilitate more flexible and efficient management of headers, particularly for scenarios involving a single header key associated with multiple values. The `multivalue_headers` field, similar to its predecessor, the `headers` field, is a key-value store. However, it can accommodate an array or list of string values for each key, instead of a single string value. This feature empowers you to represent multiple values for a single header key. Here's an example of how you might use `multivalue_headers`, using the Set-Cookie header which often has multiple values: + +``` +multivalue_headers = { + "Set-Cookie": ["sessionToken=abc123; HttpOnly; Secure", "language=en-US; Secure"], +} +``` + +In this example, Set-Cookie header has two associated values: `"sessionToken=abc123; HttpOnly; Secure"` and `"language=en-US; Secure"`. To help you understand this further, let's see how `multivalue_headers` can be used in a Tyk response plugin written in Python: + +```python +from tyk.decorators import * +from gateway import TykGateway as tyk + +@Hook +def Del_ResponseHeader_Middleware(request, response, session, metadata, spec): + # inject a new header with 2 values + new_header = response.multivalue_headers.add() + new_header.key = "Set-Cookie" + new_header.values.extend("sessionToken=abc123; HttpOnly; Secure") + new_header.values.extend("language=en-US; Secure") + + tyk.log(f"Headers content :\n {response.headers}\n----------", "info") + tyk.log(f"Multivalue Headers updated :\n {response.multivalue_headers}\n----------", "info") + + return response +``` + +In this script, we add 2 values for the `Set-Cookie` header and then log both: the traditional `headers` and the new `multivalue_headers`. This is a great way to monitor your transition to `multivalue_headers` and ensure that everything is functioning as expected. + +Please note, while the `headers` field will continue to be available and maintained for backward compatibility, we highly encourage the adoption of `multivalue_headers` for the added flexibility in handling multiple header values. + +### Go response plugins + +[Go response plugins]({{< ref "api-management/plugins/golang#creating-a-custom-response-plugin" >}}) have been available since Tyk v3.2. + +### Supported Response Plugin Languages + +See [Supported Plugins]({{< ref "api-management/plugins/overview#supported-languages" >}}) for details on which languages the response plugin is supported in. + +## Analytics Plugins + +Since Tyk 4.1.0 we have incorporated analytic plugins which enables editing or removal of all parts of analytics records and raw request and responses recorded by Tyk at the gateway level. This feature leverages existing Go plugin infrastructure. + +- Tyk receives the request. +- Tyk runs the full middleware chain, including any other plugins hooks like Pre, Post, Custom Authentication, etc. +- Tyk sends the request to your upstream API. +- The response is received and analytics plugin function is triggered before recording the hit to redis. +- Your plugin modifies the analytics record and sends it back to Tyk. +- Tyk takes the modified analytics record and record the hit in redis. + +Example analytics Go plugins can be found [here](https://github.com/TykTechnologies/tyk/blob/master/test/goplugins/test_goplugin.go#L149) + +An analytics plugin is configured using the `analytics_plugin` configuration block within an API Definition. This contains the following configuration parameters: + +- `enable`: Set to `true` to enable the plugin +- `func_name`: The name of the function representing the plugin +- `plugin_path`: The path to the source code file containing the function that implements the plugin + +{{< tabs_start >}} + +{{< tab_start "Tyk Gateway" >}} + +To enable the analytics rewriting functionality, adjust the following in API definition: + +```json +{ + "analytics_plugin": { + "enable": true, + "func_name": "", + "plugin_path": "/analytics_plugin.so" + } +} +``` + +{{< tab_end >}} + +{{< tab_start "Tyk Operator" >}} + +The example API Definition resource listed below listens on path */httpbin* and forwards requests upstream to *http://httpbin.org*. A Go Analytics Plugin is enabled for function *MaskAnalyticsData*, located within the */opt/tyk-gateway/plugins/example-plugin.so* shared object file. + +```yaml {linenos=table,hl_lines=["15-18"],linenostart=1} +apiVersion: tyk.tyk.io/v1alpha1 +kind: ApiDefinition +metadata: + name: analytics-plugin +spec: + name: httpbin-analytics-plugin + active: true + protocol: http + proxy: + listen_path: /httpbin + strip_listen_path: true + target_url: http://httpbin.org + use_keyless: true + enable_detailed_recording: true + analytics_plugin: + enable: true + func_name: MaskAnalyticsData # Replace it with function name of your plugin + plugin_path: /opt/tyk-gateway/plugins/example-plugin.so # Replace it with path of your plugin file +``` + +{{< tab_end >}} + +{{< tabs_end >}} + +
+ +## Advance Configuration + +There are two advance configuratin with plugin types: + +1. **[Per Endpoint Custom Plugin]({{< ref "" >}})** +2. **[Plugin Caching Mechanism for Authentication Plugin]({{< ref "" >}})** + + +## Per-Endpoint Custom Plugins + +Tyk's custom plugin architecture allows you to deploy custom logic that will be invoked at certain points in the [middleware chain]({{< ref "concepts/middleware-execution-order" >}}) as Tyk processes requests to your APIs. + +At the API-level, there are several points in the processing flow where custom plugins can be "hooked", as explained [here]({{< ref "api-management/plugins/plugin-types#plugin-types" >}}). Each of these will be invoked for calls to any endpoint on an API. If you want to perform custom logic only for specific endpoints, you must include selective processing logic within the plugin. + +At the endpoint-level, Tyk provides the facility to attach a custom Golang plugin at the end of the request processing chain (immediately before the API-level post-plugin is executed). + +### When to use the per-endpoint custom plugin + +##### Aggregating data from multiple services + +From a custom plugin, you can make calls out to other internal and upstream APIs. You can then aggregate and process the responses, returning a single response object to the originating client. This allows you to configure a single externally facing API to simplify interaction with multiple internal services, leaving the heavy lifting to Tyk rather than standing up an aggregation service within your stack. + +##### Enforcing custom policies + +Tyk provides a very flexible middleware chain where you can combine functions to implement the access controls you require to protect your upstream services. Of course, not all scenarios can be covered by Tyk’s standard middleware functions, but you can use a custom plugin to apply whatever custom logic you require to optimize your API experience. + +##### Dynamic Routing + +With a custom plugin you can implement complex dynamic routing of requests made to a single external endpoint on to different upstream services. The flexibility of the virtual endpoint gives access to data within the request (including the key session) and also the ability to make calls to other APIs to make decisions on the routing of the request. It can operate as a super-powered URL rewrite middleware. + +### How the per-endpoint custom plugin works + +Tyk Gateway is written using Golang. This has a flexible plugin architecture which allows for custom code to be compiled separately from the gateway and then invoked natively by the gateway. When registering a custom Go plugin in the API definition, you must provide the location of the compiled plugin and also the name of the function to be invoked within that package. + +Go plugins must therefore be [compiled]({{< ref "api-management/plugins/golang#plugin-compiler" >}}) and [loaded]({{< ref "api-management/plugins/golang#loading-custom-go-plugins-into-tyk" >}}) into the Gateway in order that the function named in the plugin configuration in the API definition can be located and executed at the appropriate stage in the request middleware processing chain. + +The custom code within the plugin has access to contextual data such as the session object and API definition. If required, it can [terminate the request]({{< ref "api-management/plugins/golang#terminating-the-request" >}}) and hence can provide a [Virtual Endpoint]({{< ref "advanced-configuration/compose-apis/virtual-endpoints" >}}) style capability using the Go language, rather than JavaScript (as supported by the virtual endpoint middleware). This can then act as a high-performance replacement for the JavaScript virtual endpoints or for cases when you want to make use of external libraries. + + + +### Using the Per-Endpoint Plugin with Tyk OAS APIs + +The [per-endpoint custom plugin]({{< ref "api-management/plugins/plugin-types#per-endpoint-custom-plugins" >}}) provides the facility to attach a custom Go plugin at the end of the request processing chain. +This plugin allows you to add custom logic to the processing flow for the specific endpoint without adding to the processing complexity of other endpoints. +It can [terminate the request]({{< ref "api-management/plugins/golang#terminating-the-request" >}}) if required, +and provides a [Virtual Endpoint]({{< ref "advanced-configuration/compose-apis/virtual-endpoints" >}}) style capability using the Go language, rather than JavaScript (as supported by the virtual endpoint middleware). + +The middleware is configured in the [Tyk OAS API Definition]({{< ref "api-management/gateway-config-tyk-oas#operation" >}}). You can do this via the Tyk Dashboard API or in the API Designer. + +If you're using the legacy Tyk Classic APIs, then check out the [Tyk Classic]({{< ref "api-management/plugins/plugin-types#using-the-per-endpoint-plugin-with-tyk-classic-apis" >}}) page. + +#### Using Tyk OAS API Definition + +The design of the Tyk OAS API Definition takes advantage of the `operationId` defined in the OpenAPI Document that declares both the path and method for which the middleware should be added. The `path` can contain wildcards in the form of any string bracketed by curly braces, for example `{user_id}`. These wildcards are so they are human readable and do not translate to variable names. Under the hood, a wildcard translates to the “match everything” regex of: `(.*)`. + +The endpoint plugin middleware (`postPlugins`) can be added to the `operations` section of the Tyk OAS Extension (`x-tyk-api-gateway`) in your Tyk OAS API Definition for the appropriate `operationId` (as configured in the `paths` section of your OpenAPI Document). + +The `postPlugins` object has the following configuration: + +- `enabled`: enable the middleware for the endpoint +- `functionName`: this is the name of the Go function that will be executed when the middleware is triggered +- `path`: the relative path to the source file containing the compiled Go code + +You can chain multiple plugin functions in an array. Tyk will process them in the order they appear in the API definition. + +For example: + +```json {hl_lines=["39-45"],linenos=true, linenostart=1} +{ + "components": {}, + "info": { + "title": "example-endpoint-plugin", + "version": "1.0.0" + }, + "openapi": "3.0.3", + "paths": { + "/anything": { + "get": { + "operationId": "anythingget", + "responses": { + "200": { + "description": "" + } + } + } + } + }, + "x-tyk-api-gateway": { + "info": { + "name": "example-endpoint-plugin", + "state": { + "active": true + } + }, + "upstream": { + "url": "http://httpbin.org/" + }, + "server": { + "listenPath": { + "value": "/example-endpoint-plugin/", + "strip": true + } + }, + "middleware": { + "operations": { + "anythingget": { + "postPlugins": [ + { + "enabled": true, + "functionName": "myUniqueFunctionName", + "path": "/middleware/myPlugin.so" + } + ] + } + } + } + } +} +``` + +In this example the per-endpoint custom plugin middleware has been configured for HTTP `GET` requests to the `/anything` endpoint. For any call made to this endpoint, Tyk will invoke the function `myUniqueFunctionName` in the file located at `/middleware/myPlugin.so`. + +The configuration above is a complete and valid Tyk OAS API Definition that you can import into Tyk to try out the per-endpoint custom plugin middleware. + +#### Using API Designer + +Adding a per-endpoint custom plugin to your API endpoints is easy when using the API Designer in the Tyk Dashboard, simply follow these steps: + +1. **Add an endpoint** + + From the **API Designer** add an endpoint that matches the path and method to which you want to apply the middleware. + + {{< img src="/img/dashboard/api-designer/tyk-oas-no-endpoints.png" alt="Tyk OAS API Designer showing no endpoints created" >}} + + {{< img src="/img/dashboard/api-designer/tyk-oas-add-endpoint.png" alt="Adding an endpoint to an API using the Tyk OAS API Designer" >}} + + {{< img src="/img/dashboard/api-designer/tyk-oas-no-middleware.png" alt="Tyk OAS API Designer showing no middleware enabled on endpoint" >}} + +2. **Select the Go Post-Plugin middleware** + + Select **ADD MIDDLEWARE** and choose **Go Post-Plugin** from the *Add Middleware* screen. + + {{< img src="/img/dashboard/api-designer/tyk-oas-go-plugin.png" alt="Adding the Go Post-Plugin middleware" >}} + +3. **Configure the middleware** + + You must provide the path to the compiled plugin and the name of the Go function that should be invoked by Tyk Gateway when the middleware is triggered. + + {{< img src="/img/dashboard/api-designer/tyk-oas-go-plugin-config.png" alt="Configuring the per-endpoint custom plugin" >}} + +4. **Save the API** + + Select **ADD MIDDLEWARE** to save the middleware configuration. Remember to select **SAVE API** to apply the changes. + +{{< note success >}} +**Note** + +You are only able to add one custom plugin to each endpoint when using the API Designer, however you can add more by editing the API definition directly in the Raw Definition editor. +{{< /note >}} + +### Using the Per-Endpoint Plugin with Tyk Classic APIs + +The [per-endpoint custom plugin]({{< ref "api-management/plugins/plugin-types#per-endpoint-custom-plugins" >}}) provides the facility to attach a custom Golang plugin at the end of the request processing chain. +This plugin allows you to add custom logic to the processing flow for the specific endpoint without adding to the processing complexity of other endpoints. +It can [terminate the request]({{< ref "api-management/plugins/golang#terminating-the-request" >}}), if required, +and hence can provide a [Virtual Endpoint]({{< ref "advanced-configuration/compose-apis/virtual-endpoints" >}}) style capability using the Go language, rather than JavaScript (as supported by the virtual endpoint middleware). + +This middleware is configured in the Tyk Classic API Definition. You can do this via the Tyk Dashboard API or in the API Designer. + +If you're using the newer Tyk OAS APIs, then check out the [Tyk OAS]({{< ref "api-management/plugins/plugin-types#using-the-per-endpoint-plugin-with-tyk-oas-apis" >}}) page. + +#### Using Tyk Classic API Definition + +To enable the middleware you must add a new `go_plugin` object to the `extended_paths` section of your API definition. + +The `go_plugin` object has the following configuration: + +- `path`: the endpoint path +- `method`: the endpoint HTTP method +- `func_name`: this is the "symbol" or function name you are calling in your Go plugin once loaded - a function can be called by one or more APIs +- `plugin_path`: the relative path of the shared object containing the function you wish to call, one or many `.so` files can be called + +You can register multiple plugin functions for a single endpoint. Tyk will process them in the order they appear in the API definition. + +For example: +```json {linenos=true, linenostart=1} +{ + "extended_paths": { + "go_plugin": [ + { + "disabled": false, + "path": "/anything", + "method": "GET", + "plugin_path": "/middleware/myPlugin.so", + "func_name": "myUniqueFunctionName" + } + ] + } +} +``` + +In this example the per-endpoint custom plugin middleware has been configured for HTTP `GET` requests to the `/anything` endpoint. For any call made to this endpoint, Tyk will invoke the function `myUniqueFunctionName` in the file located at `/middleware/myPlugin.so`. + +#### Using API Designer + +You can use the API Designer in the Tyk Dashboard to add the per-endpoint custom plugin middleware for your Tyk Classic API by following these steps. + +1. **Add an endpoint for the path and select the plugin** + + From the **Endpoint Designer** add an endpoint that matches the path for which you want to trigger the custom plugin function. Select the **Go Plugin** plugin. + + {{< img src="/img/dashboard/endpoint-designer/endpointplugin.png" alt="Selecting the middleware" >}} + +2. **Locate the middleware in the raw API definition** + + Once you have selected the middleware for the endpoint, you need to switch to the *Raw Definition* view and then locate the `go_plugin` section (you can search within the text editor window). + + {{< img src="/img/dashboard/endpoint-designer/endpointplugin_search.png" alt="Locating the middleware configuration" >}} + +3. **Configure the middleware** + + Now you can directly edit the `plugin_path` and `func_name` to locate your compiled plugin function. + + {{< img src="/img/dashboard/endpoint-designer/endpointplugin_config.png" alt="Configuring the middleware" >}} + +4. **Save the API** + + Use the *save* or *create* buttons to save the changes and activate the middleware. + +## Plugin Caching Mechanism + +The **ID extractor** is a caching mechanism that's used in combination with Tyk Plugins. It is used specifically with plugins that implement **custom authentication mechanisms**. + +We use the term `ID` to describe any key that's used for authentication purposes. + +When a custom authentication mechanism is used, every API call triggers a call to the associated middleware function, if you're using a gRPC-based plugin this translates into a gRPC call. If you're using a native plugin -like a Python plugin-, this involves a Python interpreter call. + +The ID extractor works the following rich plugins: gRPC-based plugins, Python and Lua. + +### When to use the ID Extractor? + +The main idea of the ID extractor is to reduce the number of calls made to your plugin and cache the API keys that have been already authorized by your authentication mechanism. This means that after a successful authentication event, subsequent calls will be handled by the Tyk Gateway and its Redis cache, resulting in a performance similar to the built-in authentication mechanisms that Tyk provides. + +### When does the ID Extractor Run? + +When enabled, the ID extractor runs right before the authentication step, allowing it to take control of the flow and decide whether to call your authentication mechanism or not. + +If my ID is cached by this mechanism and my plugin isn't longer called, how do I expire it? +When you implement your own authentication mechanism using plugins, you initialise the session object from your own code. The session object has a field that's used to configure the lifetime of a cached ID, this field is called `id_extractor_deadline`. See [Plugin Data Structures]({{< ref "api-management/plugins/rich-plugins#rich-plugins-data-structures" >}}) for more details. +The value of this field should be a UNIX timestamp on which the cached ID will expire, like `1507268142958`. It's an integer. + +For example, this snippet is used in a NodeJS plugin, inside a custom authentication function: + +``` +// Initialize a session state object + var session = new tyk.SessionState() + // Get the current UNIX timestamp + var timestamp = Math.floor( new Date() / 1000 ) + // Based on the current timestamp, add 60 seconds: + session.id_extractor_deadline = timestamp + 60 + // Finally inject our session object into the request object: + Obj.session = session +``` + +If you already have a plugin that implements a custom authentication mechanism, appending the `id_extractor_deadline` and setting its value is enough to activate this feature. +In the above sample, Tyk will cache the key for 60 seconds. During that time any requests that use the cached ID won't call your plugin. + +### How to enable the ID Extractor + +The ID extractor is configured on a per API basis. +The API should be a protected one and have the `enable_coprocess_auth` flag set to true, like the following definition: + +```json +{ + "name": "Test API", + "api_id": "my-api", + "org_id": "my-org", + "use_keyless": false, + "auth": { + "auth_header_name": "Authorization" + }, + "proxy": { + "listen_path": "/test-api/", + "target_url": "http://httpbin.org/", + "strip_listen_path": true + }, + "enable_coprocess_auth": true, + "custom_middleware_bundle": "bundle.zip" +} +``` + +If you're not using the Community Edition, check the API settings in the dashboard and make sure that "Custom Auth" is selected. + +The second requirement is to append an additional configuration block to your plugin manifest file, using the `id_extractor` key: + +```json +{ + "custom_middleware": { + "auth_check": { "name": "MyAuthCheck" }, + "id_extractor": { + "extract_from": "header", + "extract_with": "value", + "extractor_config": { + "header_name": "Authorization" + } + }, + "driver": "grpc" + } +} +``` + +* `extract_from` specifies the source of the ID to extract. +* `extract_with` specifies how to extract and parse the extracted ID. +* `extractor_config` specifies additional parameters like the header name or the regular expression to use, this is different for every choice, see below for more details. + + +### Available ID Extractor Sources + +#### Header Source + +Use this source to extract the key from a HTTP header. Only the name of the header is required: + +```json +{ + "id_extractor": { + "extract_from": "header", + "extract_with": "value", + "extractor_config": { + "header_name": "Authorization" + } + } +} +``` + +#### Form source + +Use this source to extract the key from a submitted form, where `param_name` represents the key of the submitted parameter: + + +```json +{ + "id_extractor": { + "extract_from": "form", + "extract_with": "value", + "extractor_config": { + "param_name": "my_param" + } + } +} +``` + + +### Available ID Extractor Modes + +#### Value Extractor + +Use this to take the value as its present. This is commonly used in combination with the header source: + +```json +{ + "id_extractor": { + "extract_from": "header", + "extract_with": "value", + "extractor_config": { + "header_name": "Authorization" + } + } +} +``` + +#### Regular Expression Extractor + +Use this to match the ID with a regular expression. This requires additional parameters like `regex_expression`, which represents the regular expression itself and `regex_match_index` which is the item index: + +```json +{ + "id_extractor": { + "extract_from": "header", + "extract_with": "regex", + "extractor_config": { + "header_name": "Authorization", + "regex_expression": "[^-]+$", + "regex_match_index": 0 + } + } +} +``` + +Using the example above, if we send a header like `prefix-d28e17f7`, given the regular expression we're using, the extracted ID value will be `d28e17f7`. + +### Example Session +Here's an example of a Session being built in GoLang custom middleware: +```{.copyWrapper} +extractorDeadline := time.Now().Add(time.Second * 5).Unix() +object.Session = &coprocess.SessionState{ + + LastUpdated: time.Now().String(), + Rate: 5, + Per: 10, + QuotaMax: int64(0), + QuotaRenews: time.Now().Unix(), + Metadata: map[string]string{ + "token": "my-unique-token", + }, + ApplyPolicies: ["5d8929d8f56e1a138f628269"], + } +``` +[source](https://github.com/TykTechnologies/tyk-grpc-go-basicauth-jwt/blob/master/main.go#L102) + +Note: When using an ID Extractor, you must set a `LastUpdated` or else token updates will not be applied. If you don't set an ID Extractor, Tyk will store session information in the cache based off the `token` field that is set in the metadata. + diff --git a/tyk-docs/content/api-management/plugins/rich-plugins.md b/tyk-docs/content/api-management/plugins/rich-plugins.md new file mode 100644 index 0000000000..891b1b0f5e --- /dev/null +++ b/tyk-docs/content/api-management/plugins/rich-plugins.md @@ -0,0 +1,3590 @@ +--- +title: "Rich Plugins" +date: 2025-01-10 +tags: [] +description: "How to manage users, teams, permissions, rbac in Tyk Dashboard" +keywords: [] +aliases: + - /plugins/supported-languages/rich-plugins + - /plugins/supported-languages/rich-plugins/rich-plugins-work + - /plugins/supported-languages/rich-plugins/rich-plugins-data-structures + - /plugins/supported-languages/rich-plugins/python/python + - /plugins/supported-languages/rich-plugins/python/custom-auth-python-tutorial + - /plugins/supported-languages/rich-plugins/python/tutorial-add-demo-plugin-api + - /plugins/supported-languages/rich-plugins/python/tyk-python-api-methods + - /plugins/supported-languages/rich-plugins/python/performance + - /plugins/supported-languages/rich-plugins/grpc + - /plugins/supported-languages/rich-plugins/grpc/write-grpc-plugin + - /plugins/supported-languages/rich-plugins/grpc/getting-started-python + - /plugins/supported-languages/rich-plugins/grpc/request-transformation-java + - /plugins/supported-languages/rich-plugins/grpc/custom-auth-dot-net + - /plugins/supported-languages/rich-plugins/grpc/custom-auth-nodejs + - /plugins/supported-languages/rich-plugins/grpc/custom-auth-python + - /plugins/supported-languages/rich-plugins/grpc/performance + - /plugins/supported-languages/rich-plugins/luajit + - /plugins/supported-languages/rich-plugins/luajit/requirements + - /plugins/supported-languages/rich-plugins/luajit/tutorial-add-demo-plugin-api + - /plugins/rich-plugins + - /plugins/supported-languages/rich-plugins + - /plugins/rich-plugins/rich-plugins-work + - /plugins/rich-plugins/rich-plugins-data-structures + - /customise-tyk/plugins/rich-plugins/rich-plugins-work + - /customise-tyk/plugins/rich-plugins/python + - /plugins/rich-plugins/python + - /customise-tyk/plugins/rich-plugins/python/custom-auth-python-tutorial + - /plugins/supported-languages/rich-plugins/python/custom-auth-python-tutorial + - /plugins/rich-plugins/python/custom-auth-python-tutorial + - /plugins/supported-languages/rich-plugins/python/tutorial-add-demo-plugin-api + - /plugins/rich-plugins/python/tutorial-add-demo-plugin-api + - /customise-tyk/plugins/rich-plugins/python/tutorial-add-demo-plugin-api + - /plugins/supported-languages/rich-plugins/python/tyk-python-api-methods + - /plugins/rich-plugins/python/tyk-python-api-methods + - /plugins/supported-languages/rich-plugins/python/performance + - /plugins/rich-plugins/python/performance + - /plugins/supported-languages/rich-plugins/luajittutorial-add-demo-plugin-api + - /plugins/rich-plugins/luajit/tutorial-add-demo-plugin-api + - /plugins/supported-languages/rich-plugins/luajitrequirements + - /plugins/rich-plugins/luajit/requirements + - /plugins/supported-languages/rich-plugins/luajit + - /plugins/rich-plugins/luajit + - /plugins/supported-languages/rich-plugins/luajit + - /plugins/supported-languages/rich-plugins/luajit/requirements + - /plugins/supported-languages/rich-plugins/luajit/tutorial-add-demo-plugin-api + - /plugins/supported-languages/rich-plugins/grpc + - /plugins/supported-languages/rich-plugins/grpc/write-grpc-plugin + - /plugins/supported-languages/rich-plugins/grpc/getting-started-python + - /plugins/supported-languages/rich-plugins/grpc/request-transformation-java + - /plugins/supported-languages/rich-plugins/grpc/custom-auth-dot-net + - /plugins/supported-languages/rich-plugins/grpc/custom-auth-nodejs + - /plugins/supported-languages/rich-plugins/grpc/custom-auth-python + - /plugins/supported-languages/rich-plugins/grpc/performance + - /plugins/rich-plugins/grpc/performance + - /plugins/rich-plugins/grpc/custom-auth-nodejs + - /customise-tyk/plugins/rich-plugins/grpc/custom-auth-dot-net/ + - /plugins/rich-plugins/grpc/custom-auth-dot-net + - /plugins/rich-plugins/grpc/request-transformation-java + - /plugins/supported-languages/rich-plugins/grpc/write-grpc-plugin + - /plugins/rich-plugins/grpc/write-grpc-plugin + - /plugins/supported-languages/rich-plugins/grpc/tutorial-add-grpc-plugin-api + - /plugins/rich-plugins/grpc/tutorial-add-grpc-plugin-api + - /customise-tyk/plugins/rich-plugins/grpc/ + - /customise-tyk/plugins/rich-plugins/grpc/ + - /plugins/rich-plugins/grpc + - /plugins/rich-plugins/grpc/grpc-plugins-tyk +--- + +## Introduction + +Rich plugins make it possible to write powerful middleware for Tyk. Tyk supports: + +* [Python]({{< ref "api-management/plugins/rich-plugins#overview" >}}) +* [gRPC]({{< ref "api-management/plugins/rich-plugins#overview-1" >}}) +* [Lua]({{< ref "api-management/plugins/rich-plugins#using-lua" >}}) + +gRPC provides the ability to write plugins using many languages including C++, Java, Ruby and C#. + +The dynamically built Tyk binaries can expose and call Foreign Function Interfaces in guest languages that extend the functionality of a gateway process. + +The plugins are able to directly call some Tyk API functions from within their guest language. They can also be configured so that they hook into various points along the standard middleware chain. + +{{< note success >}} +**Note** + +When using Python plugins, the middleware function names are set globally. So, if you include two or more plugins that implement the same function, the last declared plugin implementation of the function will be returned. We plan to add namespaces in the future. +{{< /note >}} + +## How do rich plugins work ? + +### ID Extractor & Auth Plugins + +The ID Extractor is a caching mechanism that's used in combination with Tyk Plugins. It can be used specifically with plugins that implement custom authentication mechanisms. The ID Extractor works for all rich plugins: gRPC-based plugins, Python and Lua. + +See [ID Extractor]({{< ref "api-management/plugins/plugin-types#plugin-caching-mechanism" >}}) for more details. + +### Interoperability + +This feature implements an in-process message passing mechanism, based on [Protocol Buffers](https://developers.google.com/protocol-buffers/), any supported languages should provide a function to receive, unmarshal and process this kind of messages. + +The main interoperability task is achieved by using [cgo](https://golang.org/cmd/cgo/) as a bridge between a supported language -like Python- and the Go codebase. + +Your C bridge function must accept and return a `CoProcessMessage` data structure like the one described in [`api.h`](https://github.com/TykTechnologies/tyk/blob/master/coprocess/api.h), where `p_data` is a pointer to the serialised data and `length` indicates the length of it. + +```{.copyWrapper} +struct CoProcessMessage { + void* p_data; + int length; +}; +``` + +The unpacked data will hold the actual `CoProcessObject` data structure. + +- `HookType` - the hook type (see below) +- `Request` - the HTTP request +- `Session` - the [Tyk session object]({{< ref "tyk-apis/tyk-gateway-api/token-session-object-details" >}}). +- `Metadata` - the metadata from the session data above (key/value string map). +- `Spec` - the API specification data. Currently organization ID, API ID and config_data. + +```{.copyWrapper} +type CoProcessObject struct { + HookType string + Request CoProcessMiniRequestObject + Session SessionState + Metadata map[string]string + Spec map[string]string +} +``` + +### Coprocess Dispatcher + +`Coprocess.Dispatcher` describes a very simple interface for implementing the dispatcher logic, the required methods are: `Dispatch`, `DispatchEvent` and `Reload`. + +`Dispatch` accepts a pointer to a `struct CoProcessObject` (as described above) and must return an object of the same type. This method will be called for every configured hook on every request. Traditionally this method will perform a single function call on the target language side (like `Python_DispatchHook` in `coprocess_python`), and the corresponding logic will be handled from there (mostly because different languages have different ways of loading, referencing or calling middlewares). + +`DispatchEvent` provides a way of dispatching Tyk events to a target language. This method doesn't return any variables but does receive a JSON-encoded object containing the event data. For extensibility purposes, this method doesn't use Protocol Buffers, the input is a `[]byte`, the target language will take this (as a `char`) and perform the JSON decoding operation. + +`Reload` is called when triggering a hot reload, this method could be useful for reloading scripts or modules in the target language. + +### Coprocess Dispatcher - Hooks + +This component is in charge of dispatching your HTTP requests to the custom middlewares. The list, from top to bottom, shows the order of execution. The dispatcher follows the standard middleware chain logic and provides a simple mechanism for "hooking" your custom middleware behavior, the supported hooks are: + +* **Pre**: gets executed before the request is sent to your upstream target and before any authentication information is extracted from the header or parameter list of the request. When enabled, this applies to both keyless and protected APIs. +* **AuthCheck**: gets executed as a custom authentication middleware, instead of the standard ones provided by Tyk. Use this to provide your own authentication mechanism. +* **PostKeyAuth**: gets executed right after the authentication process. +* **Post**: gets executed after the authentication, validation, throttling, and quota-limiting middleware has been executed, just before the request is proxied upstream. Use this to post-process a request before sending it to your upstream API. This is only called when using protected APIs. If you want to call a hook after the authentication but before the validation, throttling and other middleware, see **PostKeyAuth**. +* **Response**: gets executed after the upstream API replies. The arguments passed to this hook include both the request and response data. Use this to modify the HTTP response before it's sent to the client. This hook also receives the request object, the session object, the metadata and API definition associated with the request. + +{{< note success >}} +**Note** + +Response hooks are not available for native Go plugins. Python and gRPC plugins are supported. +{{< /note >}} + + +### Coprocess Gateway API + +[`coprocess_api.go`](https://github.com/TykTechnologies/tyk/tree/master/coprocess) provides a bridge between the Gateway API and C. Any function that needs to be exported should have the `export` keyword: + +```{.copyWrapper} +//export TykTriggerEvent +func TykTriggerEvent( CEventName *C.char, CPayload *C.char ) { + eventName := C.GoString(CEventName) + payload := C.GoString(CPayload) + + FireSystemEvent(tykcommon.TykEvent(eventName), EventMetaDefault{ + Message: payload, + }) +} +``` + +You should also expect a header file declaration of this function in [`api.h`](https://github.com/TykTechnologies/tyk/blob/master/coprocess/api.h), like this: + +```{.copyWrapper} +#ifndef TYK_COPROCESS_API +#define TYK_COPROCESS_API +extern void TykTriggerEvent(char* event_name, char* payload); +#endif +``` + +The language binding will include this header file (or declare the function inline) and perform the necessary steps to call it with the appropriate arguments (like an FFI mechanism could do). As a reference, this is how this could be achieved if you're building a [Cython](http://cython.org/) module: + +```{.copyWrapper} +cdef extern: + void TykTriggerEvent(char* event_name, char* payload); + +def call(): + event_name = 'my event'.encode('utf-8') + payload = 'my payload'.encode('utf-8') + TykTriggerEvent( event_name, payload ) +``` + +### Basic usage + +The intended way of using a Coprocess middleware is to specify it as part of an API Definition: + +```{.json} +"custom_middleware": { + "pre": [ + { + "name": "MyPreMiddleware", + "require_session": false + }, + { + "name": "AnotherPreMiddleware", + "require_session": false + } + ], + "post": [ + { + "name": "MyPostMiddleware", + "require_session": false + } + ], + "post_key_auth": [ + { + "name": "MyPostKeyAuthMiddleware", + "require_session": true + } + ], + "auth_check": { + "name": "MyAuthCheck" + }, + "driver": "python" +} +``` +{{< note success >}} +**Note** + +All hook types support chaining except the custom auth check (`auth_check`). +{{< /note >}} + +## Rich Plugins Data Structures + +This page describes the data structures used by Tyk rich plugins, for the following plugin drivers: + +- Python (built-in) +- Lua (built-in) +- gRPC (external, compatible with any supported [gRPC language](https://grpc.io/docs/)) + +The Tyk [Protocol Buffer definitions](https://github.com/TykTechnologies/tyk/tree/master/coprocess/proto) are intended for users to generate their own bindings using the appropriate gRPC tools for the required target language. +The remainder of this document illustrates a class diagram and explins the attributes of the protobuf messages. + +--- + +### Class Diagram + +The class diagram below illustrates the structure of the [Object](#object) message, dispatched by Tyk to a gRPC server that handles custom plugins. + +{{< img src="/img/grpc/grpc-class-diagram.svg" width="600" >}} + +--- + +### Object + +The `Coprocess.Object` data structure wraps a `Coprocess.MiniRequestObject` and `Coprocess.ResponseObject` It contains additional fields that are useful for users that implement their own request dispatchers, like the middleware hook type and name. +It also includes the session state object (`SessionState`), which holds information about the current key/user that's used for authentication. + +```protobuf +message Object { + HookType hook_type = 1; + string hook_name = 2; + MiniRequestObject request = 3; + SessionState session = 4; + map metadata = 5; + map spec = 6; + ResponseObject response = 7; +} +``` + +#### Field Descriptions + +`hook_type` +Contains the middleware hook type: pre, post, custom auth. + +`hook_name` +Contains the hook name. + +`request` +Contains the request object, see `MiniRequestObject` for more details. + +`session` +Contains the session object, see `SessionState` for more details. + +`metadata` +Contains the metadata. This is a dynamic field. + +`spec` +Contains information about API definition, including `APIID`, `OrgID` and `config_data`. + +`response` +Contains information populated from the upstream HTTP response data, for response hooks. See [ResponseObject](#responseobject) for more details. All the field contents can be modified. + +--- + +### MiniRequestObject + +The `Coprocess.MiniRequestObject` is the main request data structure used by rich plugins. It's used for middleware calls and contains important fields like headers, parameters, body and URL. A `MiniRequestObject` is part of a `Coprocess.Object`. + +```protobuf +message MiniRequestObject { + map headers = 1; + map set_headers = 2; + repeated string delete_headers = 3; + string body = 4; + string url = 5; + map params = 6; + map add_params = 7; + map extended_params = 8; + repeated string delete_params = 9; + ReturnOverrides return_overrides = 10; + string method = 11; + string request_uri = 12; + string scheme = 13; + bytes raw_body = 14; +} +``` + +#### Field Descriptions + +`headers` +A read-only field for reading headers injected by previous middleware. Modifying this field won't alter the request headers See `set_headers` and `delete_headers` for this. + +`set_headers` +This field appends the given headers (keys and values) to the request. + +`delete_headers` +This field contains an array of header names to be removed from the request. + +`body` +Contains the request body. See `ReturnOverrides` for response body modifications. + +`raw_body` +Contains the raw request body (bytes). + +`url` +The request URL. + +`params` +A read-only field that contains the request params. Modifying this value won't affect the request params. + +`add_params` +Add paramaters to the request. + +`delete_params` +This field contains an array of parameter keys to be removed from the request. + +`return_overrides` +See `ReturnOverrides` for more information. + +`method` +The request method, e.g. GET, POST, etc. + +`request_uri` +Raw unprocessed URL which includes query string and fragments. + +`scheme` +Contains the URL scheme, e.g. `http`, `https`. + +--- + +### ResponseObject + +The `ResponseObject` exists within an [object](#object) for response hooks. The fields are populated with the upstream HTTP response data. All the field contents can be modified. + +```protobuf +syntax = "proto3"; + +package coprocess; + +message ResponseObject { + int32 status_code = 1; + bytes raw_body = 2; + string body = 3; + map headers = 4; + repeated Header multivalue_headers = 5; +} + +message Header { + string key = 1; + repeated string values = 2; +} +``` + +#### Field Descriptions + +`status_code` +This field indicates the HTTP status code that was sent by the upstream. + +`raw_body` +This field contains the HTTP response body (bytes). It's always populated. + +`body` +This field contains the HTTP response body in string format. It's not populated if the `raw_body` contains invalid UTF-8 characters. + +`headers` +A map that contains the headers sent by the upstream. + +`multivalue_headers` +A list of headers, each header in this list is a structure that consists of two parts: a key and its corresponding values. +The key is a string that denotes the name of the header, the values are a list of strings that hold the content of the header, this is useful when the header has multiple associated values. +This field is available for Go, Python and Ruby since tyk v5.0.4 and 5.1.1+. + +--- + +### ReturnOverrides + +The `ReturnOverrides` object, when returned as part of a `Coprocess.Object`, overrides the response of a given HTTP request. It also stops the request flow and the HTTP request isn't passed upstream. The fields specified in the `ReturnOverrides` object are used as the HTTP response. +A sample usage for `ReturnOverrides` is when a rich plugin needs to return a custom error to the user. + +```protobuf +syntax = "proto3"; + +package coprocess; + +message ReturnOverrides { + int32 response_code = 1; + string response_error = 2; + map headers = 3; + bool override_error = 4; + string response_body = 5; +} +``` + +#### Field Descriptions + +`response_code` +This field overrides the HTTP response code and can be used for error codes (403, 500, etc.) or for overriding the response. + +`response_error` +This field overrides the HTTP response body. + +`headers` +This field overrides response HTTP headers. + +`override_error` +This setting provides enhanced customization for returning custom errors. It should be utilized alongside `response_body` for optimal effect. + +`response_body` +This field serves as an alias for `response_erro`r and holds the HTTP response body. + +--- + +### SessionState {#session-state} + +A `SessionState` data structure is created for every authenticated request and stored in Redis. It's used to track the activity of a given key in different ways, mainly by the built-in Tyk middleware like the quota middleware or the rate limiter. +A rich plugin can create a `SessionState` object and store it in the same way built-in authentication mechanisms do. This is what a custom authentication middleware does. This is also part of a `Coprocess.Object`. +Returning a null session object from a custom authentication middleware is considered a failed authentication and the appropriate HTTP 403 error is returned by the gateway (this is the default behavior) and can be overridden by using `ReturnOverrides`. + +#### Field Descriptions + +`last_check` +No longer used. + +`allowance` +No longer in use, should be the same as `rate`. + +`rate` +The number of requests that are allowed in the specified rate limiting window. + +`per` +The number of seconds that the rate window should encompass. + +`expires` +An epoch that defines when the key should expire. + +`quota_max` +The maximum number of requests allowed during the quota period. + +`quota_renews` +An epoch that defines when the quota renews. + +`quota_remaining` +Indicates the remaining number of requests within the user's quota, which is independent of the rate limit. + +`quota_renewal_rate` +The time in seconds during which the quota is valid. So for 1000 requests per hour, this value would be 3600 while `quota_max` and `quota_remaining` would be 1000. + +`access_rights` +Defined as a `map` instance, that maps the session's API ID to an [AccessDefinition](#access-definition). The AccessDefinition defines the [access rights]({{< ref "api-management/policies#setting-granular-paths-on-a-per-key-basis" >}}) for the API in terms of allowed: versions and URLs(endpoints). Each URL (endpoint) has a list of allowed methods. For further details consult the tutorials for how to create a [security policy]({{< ref "getting-started/create-security-policy" >}}) for Tyk Cloud, Tyk Self Managed and Tyk OSS platforms. + +`org_id` +The organization this user belongs to. This can be used in conjunction with the org_id setting in the API Definition object to have tokens "owned" by organizations. + +`oauth_client_id` +This is set by Tyk if the token is generated by an OAuth client during an OAuth authorization flow. + +`basic_auth_data` +This section contains a hashed representation of the basic auth password and the hashing method used. +For further details see [BasicAuthData](#basicauthdata). + +`jwt_data` +Added to sessions where a Tyk key (embedding a shared secret) is used as the public key for signing the JWT. The JWT token's KID header value references the ID of a Tyk key. See [JWTData](#jwtdata) for an example. + +`hmac_enabled` +When set to `true` this indicates generation of a [HMAC signature]({{< ref "/api-management/client-authentication#sign-requests-with-hmac" >}}) using the secret provided in `hmac_secret`. If the generated signature matches the signature provided in the *Authorization* header then authentication of the request has passed. + +`hmac_secret` +The value of the HMAC shared secret. + +`is_inactive` +Set this value to true to deny access. + +`apply_policy_id` +The policy ID that is bound to this token. + +{{< note success >}} +**Note** + +Although `apply_policy_id` is still supported, it is now deprecated. `apply_policies` is now used to list your policy IDs as an array. This supports the **[Multiple Policy]({{< ref "api-management/policies#partitioned-policy-functionality" >}})** feature introduced in the **v2.4 - 1.4** release. +{{< /note >}} + +`data_expires` +A value, in seconds, that defines when data generated by this token expires in the analytics DB (must be using Pro edition and MongoDB). + +`monitor` +Defines a [quota monitor]({{< ref "basic-config-and-security/report-monitor-trigger-events/monitors" >}}) containing a list of percentage threshold limits in descending order. These limits determine when webhook notifications are triggered for API users or an organization. Each threshold represents a percentage of the quota that, when reached, triggers a notification. See [Monitor](#monitor) for further details and an example. + +`enable_detailed_recording` +Set this value to true to have Tyk store the inbound request and outbound response data in HTTP Wire format as part of the analytics data. + +`metadata` +Metadata to be included as part of the session. This is a key/value string map that can be used in other middleware such as transforms and header injection to embed user-specific data into a request, or alternatively to query the providence of a key. + +`tags` +Tags are embedded into analytics data when the request completes. If a policy has tags, those tags will supersede the ones carried by the token (they will be overwritten). + +`alias` +As of v2.1, an Alias offers a way to identify a token in a more human-readable manner, add an Alias to a token in order to have the data transferred into Analytics later on so you can track both hashed and un-hashed tokens to a meaningful identifier that doesn't expose the security of the underlying token. + +`last_updated` +A UNIX timestamp that represents the time the session was last updated. Applicable to *Post*, *PostAuth* and *Response* plugins. When developing *CustomAuth* plugins developers should add this to the SessionState instance. + +`id_extractor_deadline` +This is a UNIX timestamp that signifies when a cached key or ID will expire. This relates to custom authentication, where authenticated keys can be cached to save repeated requests to the gRPC server. See [id_extractor]({{< ref "api-management/plugins/plugin-types#plugin-caching-mechanism" >}}) and [Auth Plugins]({{< ref "api-management/plugins/plugin-types#authentication-plugins" >}}) for additional information. + +`session_lifetime` +UNIX timestamp that denotes when the key will automatically expire. Any·subsequent API request made using the key will be rejected. Overrides the global session lifetime. See [Key Expiry and Deletion]({{< ref "/api-management/client-authentication#set-physical-key-expiry-and-deletion" >}}) for more information. + +--- + +### AccessDefinition {#access-definition} + +```protobuf +message AccessDefinition { + string api_name = 1; + string api_id = 2; + repeated string versions = 3; + repeated AccessSpec allowed_urls = 4; +} +``` + +Defined as an attribute within a [SessionState](#session-state) instance. Contains the allowed versions and URLs (endpoints) for the API that the session request relates to. Each URL (endpoint) specifies an associated list of allowed methods. See also [AccessSpec](#access-spec). + +#### Field Descriptions + +`api_name` +The name of the API that the session request relates to. + +`api_id` +The ID of the API that the session request relates to. + +`versions` +List of allowed API versions, e.g. `"versions": [ "Default" ]`. + +`allowed_urls` List of [AccessSpec](#access-spec) instances. Each instance defines a URL (endpoint) with an associated allowed list of methods. If all URLs (endpoints) are allowed then the attribute is not set. + +--- + +### AccessSpec {#access-spec} + +Defines an API's URL (endpoint) and associated list of allowed methods + +```protobuf +message AccessSpec { + string url = 1; + repeated string methods = 2; +} +``` + +#### Field Descriptions + +`url` +A URL (endpoint) belonging to the API associated with the request session. + +`methods` +List of allowed methods for the URL (endpoint), e.g. `"methods": [ "GET". "POST", "PUT", "PATCH" ]`. + +--- + +### BasicAuthData + +The `BasicAuthData` contains a hashed password and the name of the hashing algorithm used. This is represented by the `basic_auth_data` attribute in [SessionState](#session-state) message. + +```yaml +"basicAuthData": { + "password": , + "hash": +} +``` + +#### Field Descriptions + +`password` +A hashed password. + +`hash` +Name of the [hashing algorithm]({{< ref "api-management/policies#access-key-hashing" >}}) used to hash the password. + +--- + +### JWTData + +Added to [sessions](#session-state) where a Tyk key (embedding a shared secret) is used as the public key for signing the JWT. This message contains the shared secret. + +```yaml +"jwtData": { + "secret": "the_secret" +} +``` + +#### Field Descriptions + +`secret` +The shared secret. + +--- + +### Monitor {#monitor} +Added to a [session](#session-state) when [monitor quota thresholds]({{< ref "basic-config-and-security/report-monitor-trigger-events/monitors" >}}) are defined within the Tyk key. This message contains the quota percentage threshold limits, defined in descending order, that trigger webhook notification. + +```yaml +message Monitor { + repeated double trigger_limits = 1; +} +``` + +#### Field Descriptions + +`trigger_limits` +List of trigger limits defined in descending order. Each limit represents the percentage of the quota that must be reached in order for the webhook notification to be triggered. + +```yaml +"monitor": { + "trigger_limits": [80.0, 60.0, 50.0] +} +``` + +--- + +
+ + + +## Using Python + +### Overview + +#### Requirements + +Since v2.9, Tyk supports any currently stable [Python 3.x version](https://www.python.org/downloads/). The main requirement is to have the Python shared libraries installed. These are available as `libpython3.x` in most Linux distributions. + +- Python3-dev +- [Protobuf](https://pypi.org/project/protobuf/): provides [Protocol Buffers](https://developers.google.com/protocol-buffers/) support +- [gRPC](https://pypi.org/project/grpcio/): provides [gRPC](http://www.grpc.io/) support + +#### Important Note Regarding Performance + +Python plugins are [embedded](https://docs.python.org/3/extending/embedding.html) within the Tyk Gateway process. Tyk Gateway integrates with Python custom plugins via a [cgo](https://golang.org/cmd/cgo) bridge. + +`Tyk Gateway` <-> CGO <-> `Python Custom Plugin` + +In order to integrate with Python custom plugins, the *libpython3.x.so* shared object library is used to embed a Python interpreter directly in the Tyk Gateway. Further details can be found [here]({{< ref "api-management/plugins/rich-plugins#coprocess-gateway-api" >}}) + +This allows combining the strengths of both Python and Go in a single application. However, it's essential to be aware of the potential complexities and performance implications of mixing languages, as well as the need for careful memory management when working with Python objects from Go. + +The Tyk Gateway process initialises the Python interpreter using [Py_initialize](https://docs.python.org/3/c-api/init.html#c.Py_Initialize). The Python [Global Interpreter Lock (GIL)](https://docs.python.org/3/glossary.html/#term-global-interpreter-lock) allows only one thread to execute Python bytecode at a time, ensuring thread safety and simplifying memory management. While the GIL simplifies these aspects, it can limit the scalability of multi-threaded applications, particularly those with CPU-bound tasks, as it restricts parallel execution of Python code. + +In the context of custom Python plugins, API calls are queued and the Python interpreter handles requests sequentially, processing them one at a time. Subsequently, this would consume large amounts of memory, and network sockets would remain open and blocked until the API request is processed. + +#### Install the Python development packages + +{{< tabs_start >}} + +{{< tab_start "Docker" >}} +{{< note success >}} +**Note** + +Starting from Tyk Gateway version `v5.3.0`, Python is no longer bundled with the official Tyk Gateway Docker image by default, to address security vulnerabilities in the Python libraries highlighted by [Docker Scout](https://docs.docker.com/scout/). +
+Whilst Python plugins are still supported by Tyk Gateway, if you want to use them you must extend the image to add support for Python. For further details, please refer to the [release notes]({{< ref "developer-support/release-notes/gateway" >}}) for Tyk Gateway `v5.3.0`. +{{< /note >}} + +If you wish to use Python plugins using Docker, you can extend the official Tyk Gateway Docker image by adding Python to it. + +This example Dockerfile extends the official Tyk Gateway image to support Python plugins by installing python and the required modules: + +```dockerfile +ARG BASE_IMAGE +FROM ${BASE_IMAGE} AS base + +FROM python:3.11-bookworm +COPY --from=base /opt/tyk-gateway/ /opt/tyk-gateway/ +RUN pip install setuptools && pip install google && pip install 'protobuf==4.24.4' + +EXPOSE 8080 80 443 + +ENV PYTHON_VERSION=3.11 +ENV PORT=8080 + +WORKDIR /opt/tyk-gateway/ + +ENTRYPOINT ["/opt/tyk-gateway/tyk" ] +CMD [ "--conf=/opt/tyk-gateway/tyk.conf" ] +``` + +To use this, you simply run `docker build` with this Dockerfile, providing the Tyk Gateway image that you would like to extend as build argument `BASE_IMAGE`. +As an example, this command will extend Tyk Gateway `v5.3.0` to support Python plugins, generating the image `tyk-gateway-python:v5.3.0`: + +```bash +docker build --build-arg BASE_IMAGE=tykio/tyk-gateway:v5.3.0 -t tyk-gateway-python:v5.3.0 . +``` + +{{< tab_end >}} + +{{< tab_start "Ubuntu/Debian" >}} + +```apt +apt install python3 python3-dev python3-pip build-essential +``` + +#### Install the Required Python Modules + +Make sure that "pip" is available in your system, it should be typically available as "pip", "pip3" or "pipX.X" (where X.X represents the Python version): + +```pip3 +pip3 install protobuf grpcio +``` + +{{< tab_end >}} + +{{< tab_start "Red Hat or CentOS" >}} + +```yum +yum install python3-devel python3-setuptools +python3 -m ensurepip +``` + +#### Install the Required Python Modules + +Make sure that "pip" is now available in your system, it should be typically available as "pip", "pip3" or "pipX.X" (where X.X represents the Python version): + +```pip3 +pip3 install protobuf grpcio +``` + +{{< tab_end >}} + +{{< tabs_end >}} + +#### Python versions + +Newer Tyk versions provide more flexibility when using Python plugins, allowing the users to set which Python version to use. By default, Tyk will try to use the latest version available. + +To see the Python initialisation log, run the Tyk gateway in debug mode. + +To use a specific Python version, set the `python_version` flag under `coprocess_options` in the Tyk Gateway configuration file (tyk.conf). + +{{< note success >}} +**Note** + +Tyk doesn't support Python 2.x. +{{< /note >}} + +#### Troubleshooting + +To verify that the required Python Protocol Buffers module is available: + +```python3 +python3 -c 'from google import protobuf' +``` + +No output is expected from this command on successful setups. + +#### How do I write Python Plugins? + +We have created [a demo Python plugin repository](https://github.com/TykTechnologies/tyk-plugin-demo-python). + +The project implements a simple middleware for header injection, using a Pre hook (see [Tyk custom middleware hooks]({{< ref "api-management/plugins/rich-plugins#coprocess-dispatcher---hooks" >}}). A single Python script contains the code for it, see [middleware.py](https://github.com/TykTechnologies/tyk-plugin-demo-python/blob/master/middleware.py). + + +### Custom Authentication Plugin Tutorial + +#### Introduction +This tutorial will guide you through the creation of a custom authentication plugin, written in Python. +A custom authentication plugin allows you to implement your own authentication logic and override the default Tyk authentication mechanism. The sample code implements a very simple key check; currently it supports a single, hard-coded key. It could serve as a starting point for your own authentication logic. We have tested this plugin with Ubuntu 14. + +The code used in this tutorial is also available in [this GitHub repository](https://github.com/TykTechnologies/tyk-plugin-demo-python). + +#### Requirements + +* Tyk API Gateway: This can be installed using standard package management tools like Yum or APT, or from source code. See [here]({{< ref "tyk-self-managed#installation-options-for-tyk-self-managed" >}}) for more installation options. + +##### Dependencies + +* The Tyk CLI utility, which is bundled with our RPM and DEB packages, and can be installed separately from [https://github.com/TykTechnologies/tyk-cli](https://github.com/TykTechnologies/tyk-cli) +* In Tyk 2.8 the Tyk CLI is part of the gateway binary, you can find more information by running "tyk help bundle". +* Python 3.4 + +#### Create the Plugin +The first step is to create a new directory for your plugin file: + +```bash +mkdir ~/my-tyk-plugin +cd ~/my-tyk-plugin +``` + +Next you need to create a manifest file. This file contains information about our plugin file structure and how you expect it to interact with the API that will load it. +This file should be named `manifest.json` and needs to contain the following content: + +```json +{ + "file_list": [ + "middleware.py" + ], + "custom_middleware": { + "driver": "python", + "auth_check": { + "name": "MyAuthMiddleware" + } + } +} +``` + +* The `file_list` block contains the list of files to be included in the bundle, the CLI tool expects to find these files in the current working directory. +* The `custom_middleware` block contains the middleware settings like the plugin driver we want to use (`driver`) and the hooks that our plugin will expose. You use the `auth_check` for this tutorial. For other hooks see [here]({{< ref "api-management/plugins/rich-plugins#coprocess-dispatcher---hooks" >}}). +* The `name` field references the name of the function that you implement in your plugin code: `MyAuthMiddleware`. +* You add an additional file called `middleware.py`, this will contain the main implementation of our middleware. + +{{< note success >}} +**Note** + +Your bundle should always contain a file named `middleware.py` as this is the entry point file. +{{< /note >}} + +##### Contents of middleware.py + +You import decorators from the Tyk module as this gives you the `Hook` decorator, and you import [Tyk Python API helpers]({{< ref "api-management/plugins/rich-plugins#tyk-python-api-methods" >}}) + +You implement a middleware function and register it as a hook, the input includes the request object, the session object, the API meta data and its specification: + +```python +from tyk.decorators import * +from gateway import TykGateway as tyk + +@Hook +def MyAuthMiddleware(request, session, metadata, spec): + auth_header = request.get_header('Authorization') + if auth_header == '47a0c79c427728b3df4af62b9228c8ae': + tyk.log("I'm logged!", "info") + tyk.log("Request body" + request.object.body, "info") + tyk.log("API config_data" + spec['config_data'], "info") + session.rate = 1000.0 + session.per = 1.0 + metadata["token"] = "47a0c79c427728b3df4af62b9228c8ae" + return request, session, metadata +``` + + +You can modify the `manifest.json` to add as many files as you want. Files that aren't listed in the `manifest.json` file will be ignored when building the plugin bundle. + +#### Building the Plugin + +A plugin bundle is a packaged version of the plugin, it may also contain a cryptographic signature of its contents. The `-y` flag tells the Tyk CLI tool to skip the signing process in order to simplify the flow of this tutorial. For more information on the Tyk CLI tool, see [here]({{< ref "api-management/plugins/overview#plugin-bundles" >}}). + +You will use the Dockerised version of the Tyk CLI tool to bundle our package. + +First, export your Tyk Gateway version to a variable. +```bash +##### THIS MUST MATCH YOUR TYK GATEWAY VERSION +$ IMAGETAG=v3.1.2 +``` + +Then run the following commands to generate a `bundle.zip` in your current directory: +```docker +$ docker run \ + --rm -w "/tmp" -v $(pwd):/tmp \ + --entrypoint "/bin/sh" -it \ + tykio/tyk-gateway:$IMAGETAG \ + -c '/opt/tyk-gateway/tyk bundle build -y' +``` + +**Success!** + +You should now have a `bundle.zip` file in the plugin directory. + +#### Publishing the Plugin + +To allow Tyk access to the plugin bundle, you need to serve this file using a web server. For this tutorial we'll use the Python built-in HTTP server (check the official docs for additional information). This server listens on port 8000 by default. To start it use: + +`python3 -m http.server` + +When the server is started our current working directory is used as the web root path, this means that our `bundle.zip` file should be accessible from the following URL: + +`http://:8000/bundle.zip` + +The Tyk Gateway fetches and loads a plugin bundle during startup time and subsequent reloads. For updating plugins using the hot reload feature, you should use different plugin bundle names as you expect them to be used for versioning purposes, e.g. bundle-1, bundle-2, etc. +If a bundle already exists, Tyk will skip the download process and load the version that's already present. + +#### Configure Tyk + +You will need to modify the Tyk global configuration file (`tyk.conf`) to use Python plugins. The following block should be present in this file: + +```json +"coprocess_options": { + "enable_coprocess": true, + "python_path_prefix": "/opt/tyk-gateway" +}, +"enable_bundle_downloader": true, +"bundle_base_url": "http://dummy-bundle-server.com/bundles/", +"public_key_path": "/path/to/my/pubkey" +``` + +##### Options + +* `enable_coprocess`: This enables the plugin +* `python_path_prefix`: Sets the path to built-in Tyk modules, this will be part of the Python module lookup path. The value used here is the default one for most installations. +* `enable_bundle_downloader`: This enables the bundle downloader +* `bundle_base_url`: This is a base URL that will be used to download the bundle. You should replace the `bundle_base_url` with the appropriate URL of the web server that's serving your plugin bundles. For now HTTP and HTTPS are supported but we plan to add more options in the future (like pulling directly from S3 buckets). You use the URL that's exposed by the test HTTP server in the previous step. +* `public_key_path`: Modify `public_key_path` in case you want to enforce the cryptographic check of the plugin bundle signatures. If the `public_key_path` isn't set, the verification process will be skipped and unsigned plugin bundles will be loaded normally. + +#### Configure an API Definition + +There are two important parameters that you need to add or modify in the API definition. +The first one is `custom_middleware_bundle` which must match the name of the plugin bundle file. If we keep this with the default name that the Tyk CLI tool uses, it will be `bundle.zip`. + +`"custom_middleware_bundle": "bundle.zip"` + +The second parameter is specific to this tutorial, and should be used in combination with `use_keyless` to allow an API to authenticate against our plugin: + +`"use_keyless": false` +`"enable_coprocess_auth": true` + +`"enable_coprocess_auth"` will instruct the Tyk gateway to authenticate this API using the associated custom authentication function that's implemented by the plugin. + +#### Configuration via the Tyk Dashboard + +To attach the plugin to an API, From the **Advanced Options** tab in the **API Designer** enter **bundle.zip** in the **Plugin Bundle ID** field. + +{{< img src="/img/2.10/plugin_bundle_id.png" alt="Plugin Options" >}} + +You also need to modify the authentication mechanism that's used by the API. +From the **Core Settings** tab in the **API Designer** select **Use Custom Authentication (Python, CoProcess, and JSVM plugins)** from the **Authentication - Authentication Mode** drop-down list. + +{{< img src="/img/2.10/custom_auth_python.png" alt="Advanced Options" >}} + +#### Testing the Plugin + +Now you can simply make an API call against the API for which we've loaded the Python plugin. + + +##### If Running Tyk Gateway from Source + +At this point you have your test HTTP server ready to serve the plugin bundle and the configuration with all the required parameters. +The final step is to start or restart the **Tyk Gateway** (this may vary depending on how you setup Tyk). +A separate service is used to load the Tyk version that supports Python (`tyk-gateway-python`), so we need to stop the standard one first (`tyk-gateway`): + +```service +service tyk-gateway stop +service tyk-gateway-python start +``` + +From now on you should use the following command to restart the service: + +```service +service tyk-gateway-python restart +``` + +A cURL request will be enough for testing our custom authentication middleware. + +This request will trigger a bad authentication: + +```curl +curl http://:8080/my-api/my-path -H 'Authorization: badtoken' +``` + +This request will trigger a successful authentication. You are using the token that's set by your Python plugin: + +```curl +curl http://:8080/my-api/my-path -H 'Authorization: 47a0c79c427728b3df4af62b9228c8ae' +``` + +#### What's Next? + +In this tutorial you learned how Tyk plugins work. For a production-level setup we suggest the following steps: + +* Configure Tyk to use your own key so that you can enforce cryptographic signature checks when loading plugin bundles, and sign your plugin bundles! +* Configure an appropriate web server and path to serve your plugin bundles. + + +### Add Python Plugin To Your Gateway + +#### API settings + +To add a Python plugin to your API, you must specify the bundle name using the `custom_middleware_bundle` field: + +```{.json} +{ + "name": "Tyk Test API", + "api_id": "1", + "org_id": "default", + "definition": { + "location": "header", + "key": "version" + }, + "auth": { + "auth_header_name": "authorization" + }, + "use_keyless": true, + "version_data": { + "not_versioned": true, + "versions": { + "Default": { + "name": "Default", + "expires": "3000-01-02 15:04", + "use_extended_paths": true, + "extended_paths": { + "ignored": [], + "white_list": [], + "black_list": [] + } + } + } + }, + "proxy": { + "listen_path": "/quickstart/", + "target_url": "http://httpbin.org", + "strip_listen_path": true + }, + "custom_middleware_bundle": "test-bundle" +} +``` + +#### Global settings + +To enable Python plugins you need to add the following block to `tyk.conf`: + +```{.copyWrapper} +"coprocess_options": { + "enable_coprocess": true, + "python_path_prefix": "/opt/tyk-gateway" +}, +"enable_bundle_downloader": true, +"bundle_base_url": "http://dummy-bundle-server.com/bundles/", +"public_key_path": "/path/to/my/pubkey", +``` + +`enable_coprocess`: enables the rich plugins feature. + +`python_path_prefix`: Sets the path to built-in Tyk modules, this will be part of the Python module lookup path. The value used here is the default one for most installations. + +`enable_bundle_downloader`: enables the bundle downloader. + +`bundle_base_url`: is a base URL that will be used to download the bundle, in this example we have `test-bundle` specified in the API settings, Tyk will fetch the URL for your specified bundle server (in the above example): `dummy-bundle-server.com/bundles/test-bundle`. You need to create and then specify your own bundle server URL. + +`public_key_path`: sets a public key, this is used for verifying signed bundles, you may omit this if unsigned bundles are used. + +### Tyk Python API methods + +Python plugins may call these Tyk API methods: + +#### store_data(key, value, ttl) + +`store_data` sets a Redis `key` with the specified `value` and `ttl`. + +#### get_data(key) + +`get_data` retrieves a Redis `key`. + +#### trigger_event(event_name, payload) + +`trigger_event` triggers an internal Tyk event, the `payload` must be a JSON object. + +#### log(msg, level) + +`log` will log a message (`msg`) using the specified `level`. + +#### log_error(*args) + +`log_error` is a shortcut for `log`, it uses the error log level. + +### Python Performance + +These are some benchmarks performed on Python plugins. Python plugins run in a standard Python interpreter, embedded inside Tyk. + +{{< img src="/img/diagrams/pythonResponseTime.png" alt="Python Performance" >}} + +{{< img src="/img/diagrams/pythonHitRate.png" alt="Python Performance" >}} + +## Using gRPC + +### Overview + +gRPC is a very powerful framework for RPC communication across different [languages](https://www.grpc.io/docs). It was created by Google and makes heavy use of HTTP2 capabilities and the [Protocol Buffers](https://developers.google.com/protocol-buffers/) serialisation mechanism to dispatch and exchange requests between Tyk and your gRPC plugins. + +When it comes to built-in plugins, we have been able to integrate several languages like Python, Javascript & Lua in a native way: this means the middleware you write using any of these languages runs in the same process. At the time of writing, the following languages are supported: C++, Java, Objective-C, Python, Ruby, Go, C# and Node.JS. + +For supporting additional languages we have decided to integrate gRPC connections and perform the middleware operations within a gRPC server that is external to the Tyk process. Please contact us to learn more: + +
+ +{{< button_left href="https://tyk.io/contact/" color="green" content="Contact us" >}} + +Tyk has built-in support for gRPC backends, enabling you to build rich plugins using any of the gRPC supported languages. See [gRPC by language](http://www.grpc.io/docs/) for further details. + +--- + +#### Architectural overview + +An example architecture is illustrated below. + +{{< img src="/img/diagrams/diagram_docs_gRPC-plugins_why-use-it-for-plugins@2x.png" alt="Using gRPC for plugins" >}} + +Here we can see that Tyk Gateway sends requests to an external Java gRPC server to handle authentication, via a CustomAuth plugin. The flow is as follows: + +- Tyk receives a HTTP request. +- Tyk serialises the request and session into a protobuf message that is dispatched to your gRPC server. +- The gRPC server performs custom middleware operations (for example, any modification of the request object). Each plugin (Pre, PostAuthKey, Post, Response etc.) is handled as separate gRPC request. +- The gRPC server sends the request back to Tyk. +- Tyk proxies the request to your upstream API. + +--- + +#### Use cases + +Deploying an external gRPC server to handle plugins provides numerous technical advantages: + +- Allows for independent scalability of the service from the Tyk Gateway. +- Utilizes a custom-designed server tailored to address specific security concerns, effectively mitigating various security risks associated with native plugins. + +--- + +#### Limitations + +At the time of writing the following features are currently unsupported and unavailable in the serialised request: +- Client certificiates +- OAuth keys +- For graphQL APIs details concerning the *max_query_depth* is unavailable +- A request query parameter cannot be associated with multiple values + +--- + +#### Developer Resources + +The [Protocol Buffers](https://github.com/TykTechnologies/tyk/tree/master/coprocess/proto ) and [bindings](https://github.com/TykTechnologies/tyk/tree/master/coprocess/bindings) provided by Tyk should be used in order for successful ommunication between Tyk Gateway and your gRPC plugin server. Documentation for the protobuf messages is available in the [Rich Plugins Data Structures]({{< ref "api-management/plugins/rich-plugins#rich-plugins-data-structures" >}}) page. + +You can generate supporting HTML documentation using the *docs* task in the [Taskfile](https://github.com/TykTechnologies/tyk/blob/master/coprocess/proto/Taskfile.yml) file of the [Tyk repository](https://github.com/TykTechnologies/tyk). This documentation explains the protobuf messages and services that allow gRPC plugins to handle a request made to the Gateway. Please refer to the README file within the proto folder of the tyk repository for further details. + +You may re-use the bindings that were generated for our samples or generate the bindings youself for Go, Python and Ruby, as implemented by the *generate* task in the [Taskfile](https://github.com/TykTechnologies/tyk/blob/master/coprocess/proto/Taskfile.yml) file of the [Tyk repository](https://github.com/TykTechnologies/tyk). + +If you wish to generate bindings for another target language you may generate the bindings yourself. The [Protocol Buffers](https://developers.google.com/protocol-buffers/) and [gRPC documentation](http://www.grpc.io/docs) provide specific requirements and instructions for each language. + +--- + +#### What's next? + +See our [getting started]({{< ref "api-management/plugins/rich-plugins#key-concepts" >}}) guide for an explanation of how to write and configure gRPC plugins. + +--- + +### Key Concepts + +This document serves as a developer's guide for understanding the key concepts and practical steps for writing and configuring gRPC plugins for Tyk Gateway. It provides technical insights and practical guidance to seamlessly integrate Tyk plugins into your infrastructure through gRPC. The goal is to equip developers with the knowledge and tools needed to effectively utilize gRPC for enhancing Tyk Gateway functionalities. + +This comprehensive guide covers essential tasks, including: + +1. **Developing a gRPC Server:** Learn how to develop a gRPC server using [Tyk protocol buffers](https://github.com/TykTechnologies/tyk/tree/master/coprocess/proto). The gRPC server facilitates the execution of Tyk plugins, which offer custom middleware for various phases of the API request lifecycle. By integrating these plugins, developers can enable Tyk Gateway with enhanced control and flexibility in managing API requests, allowing for fine-grained customization and tailored processing at each stage of the request lifecycle. + +2. **Configuring Tyk Gateway:** Set up Tyk Gateway to communicate with your gRPC Server and, optionally, an external secured web server hosting the gRPC plugin bundle for API configurations. Configure Tyk Gateway to fetch the bundle configured for an API from the web server, enabling seamless integration with gRPC plugins. Specify connection settings for streamlined integration. + +3. **API Configuration:** Customize API settings within Tyk Gateway to configure gRPC plugin utilization. Define plugin hooks directly within the API Definition or remotely via an external web server for seamless request orchestration. Tyk plugins provide custom middleware for different phases of the API request lifecycle, enhancing control and flexibility. + +4. **API Testing:** Test that Tyk Gateway integrates with your gRPC server for the plugins configured for your API. + +--- + +#### Develop gRPC server + +Develop your gRPC server, using your preferred language, to handle requests from Tyk Gateway for each of the required plugin hooks. These hooks allow Tyk Gateway to communicate with your gRPC server to execute custom middleware at various stages of the API request lifecycle. + +##### Prerequisites + +The following prerequisites are necessary for developing a gRPC server that integrates with Tyk Gateway. + +####### Tyk gRPC Protocol Buffers + +A collection of [Protocol Buffer](https://github.com/TykTechnologies/tyk/tree/master/coprocess/proto) messages are available in the Tyk Gateway repository to allow Tyk Gateway to integrate with your gRPC server, requesting execution of plugin code. These messages establish a standard set of data structures that are serialised between Tyk Gateway and your gRPC Server. Developers should consult the [Rich Plugins Data Structures]({{< ref "api-management/plugins/rich-plugins#rich-plugins-data-structures" >}}) page for further details. + +####### Protocol Buffer Compiler + +The protocol buffer compiler, `protoc`, should be installed to generate the service and data structures in your preferred language(s) from the [Tyk gRPC Protocol Buffer](https://github.com/TykTechnologies/tyk/tree/master/coprocess/proto) files. Developers should consult the [installation](https://grpc.io/docs/protoc-installation/) documentation at [grpc.io](https://grpc.io/) for an explanation of how to install `protoc`. + +##### Generate Bindings + +Generate the bindings (service and data structures) for your target language using the `protoc` compiler. Tutorials are available at [protobuf.dev](https://protobuf.dev/getting-started/) for your target language. + +##### Implement service + +Your gRPC server should implement the *Dispatcher* service to enable Tyk Gateway to integrate with your gRPC server. The Protocol Buffer definition for the *Dispatcher* service is listed below: + +```protobuf +service Dispatcher { + rpc Dispatch (Object) returns (Object) {} + rpc DispatchEvent (Event) returns (EventReply) {} +} +``` + +The *Dispatcher* service contains two RPC methods, *Dispatch* and *DispatchEvent*. Dispatch handles a requests made by Tyk Gateway for each plugin configured in your API. DispatchEvent receives notification of an event. + +Your *Dispatch* RPC should handle the request made by Tyk Gateway, implementing custom middleware for the intended plugin hooks. Each plugin hook allows Tyk Gateway to communicate with your gRPC server to execute custom middleware at various stages of the API request lifecycle, such as Pre, PostAuth, Post, Response etc. The Tyk Protocol Buffers define the [HookType](https://github.com/TykTechnologies/tyk/blob/master/coprocess/proto/coprocess_common.proto) enumeration to inspect the type of the intended gRPC plugin associated with the request. This is accessible as an attribute on the *Object* message, e.g. *object_message_instance.hook_type*. + +##### Developer resources + +Consult the [Tyk protocol buffers](https://github.com/TykTechnologies/tyk/tree/master/coprocess/proto) for the definition of the service and data structures that enable integration of Tyk gateway with your gRPC server. Tyk provides pre-generated [bindings](https://github.com/TykTechnologies/tyk/tree/master/coprocess/bindings) for C++, Java, Python and Ruby. + +Example tutorials are available that explain how to generate the protobuf bindings and implement a server for [Java]({{< ref "api-management/plugins/rich-plugins#create-a-request-transformation-plugin-with-java" >}}), [.NET]({{< ref "api-management/plugins/rich-plugins#create-custom-authentication-plugin-with-net" >}}) and [NodeJS]({{< ref "api-management/plugins/rich-plugins#create-custom-authentication-plugin-with-net" >}}). + +Tyk Github repositories are also available with examples for [Ruby](https://github.com/TykTechnologies/tyk-plugin-demo-ruby) and [C#/.NET](https://github.com/TykTechnologies/tyk-plugin-demo-dotnet) + +--- + +#### Configure Tyk Gateway + +Configure Tyk Gateway to issue requests to your gRPC server and optionally, specify the URL of the web server that will serve plugin bundles. + +##### Configure gRPC server + +Modify the root of your `tyk.conf` file to include the *coprocess_options* section, similar to that listed below: + +```yaml +"coprocess_options": { + "enable_coprocess": true, + "coprocess_grpc_server": "tcp://127.0.0.1:5555", + "grpc_authority": "localhost", + "grpc_recv_max_size": 100000000, + "grpc_send_max_size": 100000000 +}, +``` + +A gRPC server can configured under the `coprocess_options` section as follows: + +- `enable_coprocess`: Enables the rich plugins feature. +- `coprocess_grpc_server`: Specifies the gRPC server URL, in this example we're using TCP. Tyk will attempt a connection on startup and keep reconnecting in case of failure. +- `grpc_recv_max_size`: Specifies the message size supported by the gateway gRPC client, for receiving gRPC responses. +- `grpc_send_max_size`: Specifies the message size supported by the gateway gRPC client for sending gRPC requests. +- `grpc_authority`: The `authority` header value, defaults to `localhost` if omitted. Allows configuration according to [RFC 7540](https://datatracker.ietf.org/doc/html/rfc7540#section-8.1.2.3). + +When using gRPC plugins, Tyk acts as a gRPC client and dispatches requests to your gRPC server. gRPC libraries usually set a default maximum size, for example, the official gRPC Java library establishes a 4 +MB message size [https://jbrandhorst.com/post/grpc-binary-blob-stream/](https://jbrandhorst.com/post/grpc-binary-blob-stream/). + +Configuration parameters are available for establishing a message size in both directions (send and receive). For most use cases and especially if you're dealing with multiple hooks, where the same request object is dispatched, it is recommended to set both values to the same size. + +##### Configure Web server (optional) + +Tyk Gateway can be configured to download the gRPC plugin configuration for an API from a web server. For further details related to the concept of bundling plugins please refer to [plugin bundles]({{< ref "api-management/plugins/overview#plugin-bundles" >}}). + +```yaml +"enable_bundle_downloader": true, +"bundle_base_url": "https://my-bundle-server.com/bundles/", +"public_key_path": "/path/to/my/pubkey", +``` + +The following parameters can be configured: +- `enable_bundle_downloader`: Enables the bundle downloader to download bundles from a webserver. +- `bundle_base_url`: Base URL from which to serve bundled plugins. +- `public_key_path`: Public key for bundle verification (optional) + +The `public_key_path` value is used for verifying signed bundles, you may omit this if unsigned bundles are used. + +--- + +#### Configure API + +Plugin hooks for your APIs in Tyk can be configured either by directly specifying them in a configuration file on the Gateway server or by hosting the configuration externally on a web server. This section explains how to configure gRPC plugins for your API endpoints on the local Gateway or remotely from an external secured web server. + +##### Local + +This section provides examples for how to configure gRPC plugin hooks, locally within an API Definition. Examples are provided for Tyk Gateway and Tyk Operator. + +###### Tyk Gateway + +For configurations directly embedded within the Tyk Gateway, plugin hooks can be defined within your API Definition. An example snippet from a Tyk Classic API Definition is provided below: + +```yaml +"custom_middleware": { + "pre": [ + {"name": "MyPreMiddleware"} + ], + "post": [ + {"name": "MyPostMiddleware"} + ], + "auth_check": { + "name": "MyAuthCheck" + }, + "driver": "grpc" +} +``` + +For example, a Post request plugin hook has been configured with name `MyPostMiddleware`. Before the request is sent upstream Tyk Gateway will serialize the request into a [Object protobuf message]({{< ref "api-management/plugins/rich-plugins#object" >}}) with the `hook_name` property set to `MyPostMiddleware` and the `hook_type` property set to `Post`. This message will then then be dispatched to the gRPC server for processing before the request is sent upstream. + +
+{{< note success >}} +**Note** + +Ensure the plugin driver is configured as type *grpc*. Tyk will issue a request to your gRPC server for each plugin hook that you have configured. +{{< /note >}} + +###### Tyk Operator + +The examples below illustrate how to configure plugin hooks for an API Definition within Tyk Operator. + +Setting the `driver` configuring parameter to `gRPC` instructs Tyk Gateway to issue a request to your gRPC server for each plugin hook that you have configured. + +**Pre plugin hook example** + +In this example we can see that a `custom_middleware` configuration block has been used to configure a gRPC Pre request plugin hook with name `HelloFromPre`. Before any middleware is executed Tyk Gateway will serialize the request into a [Object protobuf message]({{< ref "api-management/plugins/rich-plugins#object" >}}) with the `hook_name` property set to `HelloFromPre` and the `hook_type` property set to `Pre`. This message will then then be dispatched to the gRPC server. + +```yaml {linenos=table,hl_lines=["14-18"],linenostart=1} +apiVersion: tyk.tyk.io/v1alpha1 +kind: ApiDefinition +metadata: + name: httpbin-grpc-pre +spec: + name: httpbin-grpc-pre + use_keyless: true + protocol: http + active: true + proxy: + target_url: http://httpbin.default.svc:8000 + listen_path: /httpbin-grpc-pre + strip_listen_path: true + custom_middleware: + driver: grpc + pre: + - name: HelloFromPre + path: "" +``` + +**Post plugin hook example** + +In the example we can see that a `custom_middleware` configuration block has been used to configure a gRPC Post plugin with name `HelloFromPost`. + +Before the request is sent upstream Tyk Gateway will serialize the request and session details into a [Object protobuf message]({{< ref "api-management/plugins/rich-plugins#object" >}}) with the `hook_name` property set to `HelloFromPost` and the `hook_type` property set to `Post`. This message will then then be dispatched to the gRPC server for processing before the request is sent upstream. + +```yaml {linenos=table,hl_lines=["14-18"],linenostart=1} +apiVersion: tyk.tyk.io/v1alpha1 +kind: ApiDefinition +metadata: + name: httpbin-grpc-post +spec: + name: httpbin-grpc-post + use_keyless: true + protocol: http + active: true + proxy: + target_url: http://httpbin.default.svc:8000 + listen_path: /httpbin-grpc-post + strip_listen_path: true + custom_middleware: + driver: grpc + post: + - name: HelloFromPost + path: "" +``` + +##### Remote + +It is possible to configure your API so that it downloads a bundled configuration of your plugins from an external webserver. The bundled plugin configuration is contained within a zip file. + +A gRPC plugin bundle is similar to the [standard bundling mechanism]({{< ref "api-management/plugins/overview#plugin-bundles" >}}). The standard bundling mechanism zips the configuration and plugin source code, which will be executed by Tyk. Conversely, a gRPC plugin bundle contains only the configuration (`manifest.json`), with plugin code execution being handled independently by the gRPC server. + +Bundling a gRPC plugin requires the following steps: +- Create a `manifest.json` that contains the configuration of your plugins +- Build a zip file that bundles your plugin +- Upload the zip file to an external secured webserver +- Configure your API to download your plugin bundle + +###### Create the manifest file + +The `manifest.json` file specifies the configuration for your gRPC plugins. An example `manifest.json` is listed below: + +```yaml +{ + "file_list": [], + "custom_middleware": { + "pre": [{"name": "MyPreMiddleware"}], + "post": [{"name": "MyPostMiddleware"}], + "auth_check": {"name": "MyAuthCheck"}, + "driver": "grpc" + }, + "checksum": "", + "signature": "" +} +``` + +{{< note sucess >}} +**Note** + +The source code files, *file_list*, are empty for gRPC plugins. Your gRPC server contains the source code for handling plugins. +{{< /note >}} + +###### Build plugin bundle + +A plugin bundle can be built using the Tyk Gateway binary and should only contain the `manifest.json` file: + +```bash +tyk bundle build -output mybundle.zip -key mykey.pem +``` + +The example above generates a zip file, name `mybundle.zip`. The zip file is signed with key `mykey.pem`. + +The resulting bundle file should then be uploaded to the webserver that hosts your plugin bundles. + +###### Configure API + +####### Tyk Gateway + +To add a gRPC plugin to your API definition, you must specify the bundle file name within the `custom_middleware_bundle` field: + +```yaml +{ + "name": "Tyk Test API", + ... ++ "custom_middleware_bundle": "mybundle.zip" +} +``` + +The value of the `custom_middleware_bundle` field will be used in combination with the gateway settings to construct a bundle URL. For example, if Tyk Gateway is configured with a webserver base URL of https://my-bundle-server.com/bundles/ then an attempt would be made to download the bundle from https://my-bundle-server.com/bundles/mybundle.zip. + +####### Tyk Operator + + Currently this feature is not yet documented with a Tyk Operator example for configuring an API to use plugin bundles. For further details please reach out and contact us on the [community support forum](https://community.tyk.io). + +--- + +#### Test your API Endpoint + +It is crucial to ensure the security and reliability of your gRPC server. As the developer, it is your responsibility to verify that your gRPC server is secured and thoroughly tested with appropriate test coverage. Consider implementing unit tests, integration tests and other testing methodologies to ensure the robustness of your server's functionality and security measures. This step ensures that the Tyk Gateway properly communicates with your gRPC server and executes the custom logic defined by the plugin hooks. + +Test the API endpoint using tools like *Curl* or *Postman*. Ensure that your gRPC server is running and the gRPC plugin(s) are functioning. An example using *Curl* is listed below: + +```bash +curl -X GET https://www.your-gateway-server.com:8080/api/path +``` + +Replace `https://www.your-gateway-server.com:8080/api/path` with the actual endpoint of your API. + +--- + +#### Summary + +This guide has explained the key concepts and processes for writing gRPC plugins that integrate with Tyk Gateway. The following explanations have been given: + +- Prerequisites for developing a gRPC server for your target language. +- The *Dispatcher* service interface. +- How to configure Tyk Gateway to integrate with your gRPC server. +- How to configure Tyk Gateway with an optional external web server for fetching plugin configuration. +- How to configure gRPC plugins for your APIs. +- How to test your API integration with your gRPC server using curl. + +--- + +#### What's Next? + +- Consult the [Protocol Buffer messages]({{< ref "api-management/plugins/rich-plugins#rich-plugins-data-structures" >}}) that Tyk Gateway uses when making a request to a gRPC server. +- Visit tutorial guides that explain how to implement a [Java]({{< ref "api-management/plugins/rich-plugins#create-a-request-transformation-plugin-with-java" >}}), [.NET]({{< ref "api-management/plugins/rich-plugins#create-custom-authentication-plugin-with-net" >}}) and [NodeJS]({{< ref "api-management/plugins/rich-plugins#create-custom-authentication-plugin-with-net" >}}) gRPC server. +- Visit our [plugins hub]({{< ref "api-management/plugins/overview#plugins-hub" >}}) to explore further gRPC development examples and resources. + +--- + + + + +### Getting Started: Creating A Python gRPC Server + +In the realm of API integration, establishing seamless connections between services is paramount. + +Understanding the fundamentals of gRPC server implementation is crucial, especially when integrating with a Gateway solution like Tyk. This guide aims to provide practical insights into this process, starting with the basic principles of how to implement a Python gRPC server that integrates with Tyk Gateway. + +#### Objectives + +By the end of this guide, you will be able to implement a gRPC server that will integrate with Tyk Gateway, setting the stage for further exploration in subsequent parts: + +- Establishing the necessary tools, Python libraries and gRPC service definition for implementing a gRPC server that integrates with Tyk Gateway. +- Developing a basic gRPC server that echoes the request payload to the console, showcasing the core principles of integration. +- Configuring Tyk Gateway to interact with our gRPC server, enabling seamless communication between the two services. + +Before implementing our first gRPC server it is first necessary to understand the service interface that defines how Tyk Gateway integrates with a gRPC server. + + +#### Tyk Dispatcher Service + +The *Dispatcher* service, defined in the [coprocess_object.proto](https://github.com/TykTechnologies/tyk/blob/master/coprocess/proto/coprocess_object.proto) file, contains the *Dispatch* RPC method, invoked by Tyk Gateway to request remote execution of gRPC plugins. Tyk Gateway dispatches accompanying data relating to the original client request and session. The service definition is listed below: + +```protobuf +service Dispatcher { + rpc Dispatch (Object) returns (Object) {} + rpc DispatchEvent (Event) returns (EventReply) {} +} +``` + +On the server side, we will implement the *Dispatcher* service methods and a gRPC server to handle requests from Tyk Gateway. The gRPC infrastructure decodes incoming requests, executes service methods and encodes service responses. + +Before we start developing our gRPC server we need to setup our development environment with the supporting libraries and tools. + + +#### Prerequisites + +Firstly, we need to download the [Tyk Protocol Buffers](https://github.com/TykTechnologies/tyk/tree/master/coprocess/proto) and install the Python protoc compiler. + +We are going to use the *protoc* compiler to generate the supporting classes and data structures to implement the *Dispatcher* service. + + +##### Tyk Protocol Buffers + +Issue the following command to download and extract the Tyk Protocol Buffers from the Tyk GitHub repository: + +```bash +curl -sL "https://github.com/TykTechnologies/tyk/archive/master.tar.gz " -o tyk.tar.gz && \ + mkdir tyk && \ + tar -xzvf tyk.tar.gz --strip-components=1 -C tyk && \ + mv tyk/coprocess/proto/* . && \ + rm -r tyk tyk.tar.gz +``` + +##### Install Dependencies + +We are going to setup a Python virtual environment and install some supporting dependencies. Assuming that you have Python [virtualenv](https://virtualenv.pypa.io/en/latest/installation.html) already installed, then issue the following commands to setup a Python virtual environment containing the grpcio and grpcio-tools libraries: + +```bash +python3 -m venv .venv +source .venv/bin/activate +pip install –upgrade pip +pip install grpcio grpcio-tools grpcio-reflection +``` + +The [grpcio](https://pypi.org/project/grpcio/) library offers essential functionality to support core gRPC features such as message serialisation and deserialisation. The [grpcio-tools](https://pypi.org/project/grpcio-tools/) library provides the Python *protoc* compiler that we will use to generate the supporting classes and data structures to implement our gRPC server. The [grpcio-reflection](https://pypi.org/project/grpcio-reflection/) library allows clients to query information about the services and methods provided by a gRPC server at runtime. It enables clients to dynamically discover available services, their RPC methods, in addition to the message types and field names associated with those methods. + + +##### Install grpcurl + +Follow the [installation instructions](https://github.com/fullstorydev/grpcurl?tab=readme-ov-file#installation) to install grpcurl. We will use grpcurl to send test requests to our gRPC server. + + +##### Generate Python Bindings + +We are now able to generate the Python classes and data structures to allow us to implement our gRPC server. To accomplish this we will use the Python *protoc* command as listed below: + +```bash +python -m grpc_tools.protoc --proto_path=. --python_out=. --grpc_python_out=. *.proto +``` + +This compiles the Protocol Buffer files (*.proto) from the current working directory and generates the Python classes representing the Protocol Buffer messages and services. A series of *.py* files should now exist in the current working directory. We are interested in the *coprocess_object_pb2_grpc.py* file, containing a default implementation of *Tyk’s Dispatcher* service. + +Inspect the generated Python file, *coprocess_object_pb2_grpc.py*, containing the *DispatcherServicer* class: + +```python +class DispatcherServicer(object): + """ GRPC server interface, that must be implemented by the target language """ + def Dispatch(self, request, context): + """ Accepts and returns an Object message """ + context.set_code(grpc.StatusCode.UNIMPLEMENTED) + context.set_details('Method not implemented!') + raise NotImplementedError('Method not implemented!') + def DispatchEvent(self, request, context): + """ Dispatches an event to the target language """ + context.set_code(grpc.StatusCode.UNIMPLEMENTED) + context.set_details('Method not implemented!') + raise NotImplementedError('Method not implemented!') +``` + +This superclass contains a default stub implementation for the **Dispatch** and **DispatchEvent** RPC methods, each defining request and context parameters: + +The *request* parameter allows our server to access the message payload sent by Tyk Gateway. We can use this data, pertaining to the request and session, to process and generate a response. + +The *context* parameter provides additional information and functionalities related to the RPC call, such as timeout limits, cancelation signals etc. This is a [grpc.ServicerContext](https://grpc.github.io/grpc/python/grpc.html#grpc.ServicerContext) or a [grpc.aio.ServicerContext](https://grpc.github.io/grpc/python/grpc_asyncio.html#grpc.aio.ServicerContext), object depending upon whether a synchronous or AsyncIO gRPC server is implemented. + +In the next step we will implement a subclass that will handle requests made by Tyk Gateway for remote execution of custom plugins. + + +#### Implement Dispatcher Service + +We will now develop the *Dispatcher* service, adding implementations of the *Dispatch* and *DispatchEvent* methods, to allow our gRPC server to integrate with Tyk Gateway. Before we continue, create a file, *async_server.py*, within the same folder as the generated Protocol Buffer (.proto) files. + + +##### Dispatch + +Our implementation of the Dispatch RPC method will deserialize the request payload and output to the console as JSON format. This serves as a useful development and debugging aid, allowing inspection of the request and session state dispatched by Tyk Gateway to our gRPC server. + +Copy and paste the following source code into the *async_server.py* file. Notice that we have used type hinting to aid readability. The type hints are located within the type hint files (.pyi) we generated with the protoc compiler. + + +```python +import asyncio +import grpc +import json +import signal +import logging +from google.protobuf.json_format import MessageToJson +from grpc_reflection.v1alpha import reflection +import coprocess_object_pb2_grpc +import coprocess_object_pb2 +from coprocess_common_pb2 import HookType +from coprocess_session_state_pb2 import SessionState +class PythonDispatcher(coprocess_object_pb2_grpc.DispatcherServicer): + async def Dispatch( + self, object: coprocess_object_pb2.Object, context: grpc.aio.ServicerContext + ) -> coprocess_object_pb2.Object: + logging.info(f"STATE for {object.hook_name}\n{MessageToJson(object)}\n") + if object.hook_type == HookType.Pre: + logging.info(f"Pre plugin name: {object.hook_name}") + logging.info(f"Activated Pre Request plugin from API: {object.spec.get('APIID')}") + elif object.hook_type == HookType.CustomKeyCheck: + logging.info(f"CustomAuth plugin: {object.hook_name}") + logging.info(f"Activated CustomAuth plugin from API: {object.spec.get('APIID')}") + elif object.hook_type == HookType.PostKeyAuth: + logging.info(f"PostKeyAuth plugin name: {object.hook_name}") + logging.info(f"Activated PostKeyAuth plugin from API: {object.spec.get('APIID')}") + elif object.hook_type == HookType.Post: + logging.info(f"Post plugin name: {object.hook_name}") + logging.info(f"Activated Post plugin from API: {object.spec.get('APIID')}") + elif object.hook_type == HookType.Response: + logging.info(f"Response plugin name: {object.hook_name}") + logging.info(f"Activated Response plugin from API: {object.spec.get('APIID')}") + logging.info("--------\n") + return object +``` + +Our *Dispatch* RPC method accepts the two parameters, *object* and *context*. The object parameter allows us to inspect the state and session of the request object dispatched by Tyk Gateway, via accessor methods. The *context* parameter can be used to set timeout limits etc. associated with the RPC call. + +The important takeaways from the source code listing above are: + +- The [MessageToJson](https://googleapis.dev/python/protobuf/latest/google/protobuf/json_format.html#google.protobuf.json_format.MessageToJson) function is used to deserialize the request payload as JSON. +- In the context of custom plugins we access the *hook_type* and *hook_name* attributes of the *Object* message to determine which plugin to execute. +- The ID of the API associated with the request is accessible from the spec dictionary, *object.spec.get('APIID')*. + +An implementation of the *Dispatch* RPC method must return the object payload received from Tyk Gateway. The payload can be modified by the service implementation, for example to add or remove headers and query parameters before the request is sent upstream. + + +##### DispatchEvent + +Our implementation of the *DispatchEvent* RPC method will deserialize and output the event payload as JSON. Append the following source code to the *async_server.py* file: + +```python + async def DispatchEvent( + self, event: coprocess_object_pb2.Event, context: grpc.aio.ServicerContext + ) -> coprocess_object_pb2.EventReply: + event = json.loads(event.payload) + http://logging.info (f"RECEIVED EVENT: {event}") + return coprocess_object_pb2.EventReply() +``` + +The *DispatchEvent* RPC method accepts the two parameters, *event* and *context*. The event parameter allows us to inspect the payload of the event dispatched by Tyk Gateway. The context parameter can be used to set timeout limits etc. associated with the RPC call. + +The important takeaways from the source code listing above are: + +- The event data is accessible from the *payload* attribute of the event parameter. +- An implementation of the *DispatchEvent* RPC method must return an instance of *coprocess_object_pb2.EventReply*. + + +#### Create gRPC Server + +Finally, we will implement an AsyncIO gRPC server to handle requests from Tyk Gateway to the *Dispatcher* service. We will add functions to start and stop our gRPC server. Finally, we will use *grpcurl* to issue a test payload to our gRPC server to test that it is working. + + +##### Develop gRPC Server + +Append the following source code from the listing below to the *async_server.py* file: + +```python +async def serve() -> None: + server = grpc.aio.server() + coprocess_object_pb2_grpc.add_DispatcherServicer_to_server( + PythonDispatcher(), server + ) + listen_addr = "[::]:50051" + SERVICE_NAMES = ( + coprocess_object_pb2.DESCRIPTOR.services_by_name["Dispatcher"].full_name, + reflection.SERVICE_NAME, + ) + + reflection.enable_server_reflection(SERVICE_NAMES, server) + server.add_insecure_port(listen_addr) + + logging.info ("Starting server on %s", listen_addr) + + await server.start() + await server.wait_for_termination() + +async def shutdown_server(server) -> None: + http://logging.info ("Shutting down server...") + await server.stop(None) +``` + +The *serve* function starts the gRPC server, listening for requests on port 50051 with reflection enabled. + +Clients can use reflection to list available services, obtain their RPC methods and retrieve their message types and field names dynamically. This is particularly useful for tooling and debugging purposes, allowing clients to discover server capabilities without prior knowledge of the service definitions. + +{{< note success >}} + +**note** + +A descriptor is a data structure that describes the structure of the messages, services, enums and other elements defined in a .proto file. The purpose of the descriptor is primarily metadata: it provides information about the types and services defined in the protocol buffer definition. The *coprocess_object_pb2.py* file that we generated using *protoc* contains a DESCRIPTOR field that we can use to retrieve this metadata. For further details consult the documentation for the Google's protobuf [FileDescriptor](https://googleapis.dev/python/protobuf/latest/google/protobuf/descriptor.html#google.protobuf.descriptor.FileDescriptor.services_by_name) class. + +{{< /note >}} + +The *shutdown_server* function stops the gRPC server via the *stop* method of the server instance. + +The key takeaways from the source code listing above are: + +- An instance of a gRPC server is created using *grpc.aio.server()*. +- A service implementation should be registered with the gRPC server. We register our *PythonDispatcher* class via *coprocess_object_pb2_grpc.add_DispatcherServicer_to_server(PythonDispatcher(), server)*. +- Reflection can be enabled to allow clients to dynamically discover the services available at a gRPC server. We enabled our *Dispatcher* service to be discovered via *reflection.enable_server_reflection(SERVICE_NAMES, server)*. SERVICE_NAMES is a tuple containing the full names of two gRPC services: the *Dispatcher* service obtained by using the DESCRIPTOR field within the *coprocess_object_pb2* module and the other being the standard reflection service. +- The server instance should be started via invoking and awaiting the *start* and *wait_for_termination* methods of the server instance. +- A port may be configured for the server. In this example we configured an insecure port of 50051 on the server instance via the [add_insecure_port function](https://grpc.github.io/grpc/python/grpc.html#grpc.Server.add_insecure_port). It is also possible to add a secure port via the [add_secure_port](https://grpc.github.io/grpc/python/grpc.html#grpc.Server.add_secure_port) method of the server instance, which accepts the port number in addition to an SSL certificate and key to enable TLS encryption. +- The server instance can be stopped via its stop method. + +Finally, we will allow our server to terminate upon receipt of SIGTERM and SIGINT signals. To achieve this, append the source code listed below to the *async_server.py* file. + +```python +def handle_sigterm(sig, frame) -> None: + asyncio.create_task(shutdown_server(server)) + +async def handle_sigint() -> None: + loop = asyncio.get_running_loop() + for sig in (signal.SIGINT, signal.SIGTERM): + loop.add_signal_handler(sig, loop.stop) + +if __name__ == '__main__': + logging.basicConfig(level=logging.INFO) + server = None + signal.signal(signal.SIGTERM, handle_sigterm) + try: + asyncio.get_event_loop().run_until_complete(serve()) + except KeyboardInterrupt: + pass +``` + + +##### Start gRPC Server + +Issue the following command to start the gRPC server: + +```bash +python3 -m async_server +``` + +A message should be output on the console, displaying the port number and confirming that the gRPC server has started. + + +##### Test gRPC Server + +To test our gRPC server is working, issue test requests to the *Dispatch* and *DispatchEvent* methods, using *grpcurl*. + + +####### Send Dispatch Request + +Use the *grpcurl* command to send a test dispatch request to our gRPC server: + +```bash +grpcurl -plaintext -d '{ + "hookType": "Pre", + "hookName": "MyPreCustomPluginForBasicAuth", + "request": { + "headers": { + "User-Agent": "curl/8.1.2", + "Host": "tyk-gateway.localhost:8080", + "Authorization": "Basic ZGV2QHR5ay5pbzpwYXN0cnk=", + "Accept": "*/*" + }, + "url": "/basic-authentication-valid/get", + "returnOverrides": { + "responseCode": -1 + }, + "method": "GET", + "requestUri": "/basic-authentication-valid/get", + "scheme": "https" + }, + "spec": { + "bundle_hash": "d41d8cd98f00b204e9800998ecf8427e", + "OrgID": "5e9d9544a1dcd60001d0ed20", + "APIID": "04e911d3012646d97fcdd6c846fafc4b" + } +}' localhost:50051 coprocess.Dispatcher/Dispatch +``` + +Inspect the console output of your gRPC server. It should echo the payload that you sent in the request. + + +####### Send DispatchEvent Request + +Use the grpcurl command to send a test event payload to our gRPC server: + +```bash +grpcurl -plaintext -d '{"payload": "{\"event\": \"test\"}"}' localhost:50051 coprocess.Dispatcher/DispatchEvent +``` + +Inspect the console output of your gRPC server. It should display a log similar to that shown below: + +```bash +INFO:root:RECEIVED EVENT: {'event': 'test'} +``` + +The response received from the server should be an empty event reply, similar to that shown below: + +```bash +grpcurl -plaintext -d '{"payload": "{\"event\": \"test\"}"}' localhost:50051 coprocess.Dispatcher/DispatchEvent +{} +``` + +At this point we have tested, independently of Tyk Gateway, that our gRPC Server can handle an example request payload for gRPC plugin execution. In the next section we will create a test environment for testing that Tyk Gateway integrates with our gRPC server for API requests. + + +#### Configure Test Environment + +Now that we have implemented and started a gRPC server, Tyk Gateway needs to be configured to integrate with it. To achieve this we will enable the coprocess feature and configure the URL of the gRPC server. + +We will also create an API so that we can test that Tyk Gateway integrates with our gRPC server. + + +##### Configure Tyk Gateway + +Within the root of the *tyk.conf* file, add the following configuration, replacing host and port with values appropriate for your environment: + +```yaml +"coprocess_options": { + "enable_coprocess": true, + "coprocess_grpc_server": "tcp://host:port" +} +``` + +Alternatively, the following environment variables can be set in your .env file: + +```bash +TYK_GW_COPROCESSOPTIONS_ENABLECOPROCESS=true +TYK_GW_COPROCESSOPTIONS_COPROCESSGRPCSERVER=tcp://host:port +``` + +Replace host and port with values appropriate for your environment. + + +##### Configure API + +Before testing our gRPC server we will create and configure an API with 2 plugins: + +- **Pre Request**: Named *MyPreRequestPlugin*. +- **Response**: Named *MyResponsePlugin* and configured so that Tyk Gateway dispatches the session state with the request. + +Each plugin will be configured to use the *grpc* plugin driver. + +Tyk Gateway will forward details of an incoming request to the gRPC server, for each of the configured API plugins. + + +####### Tyk Classic API + +gRPC plugins can be configured within the *custom_middleware* section of the Tyk Classic ApiDefinition, as shown in the listing below: + +```yaml +{ + "created_at": "2024-03-231T12:49:52Z", + "api_model": {}, + "api_definition": { + ... + ... + "custom_middleware": { + "pre": [ + { + "disabled": false, + "name": "MyPreRequestPlugin", + "path": "", + "require_session": false, + "raw_body_only": false + } + ], + "post": [], + "post_key_auth": [], + "auth_check": { + "disabled": false, + "name": "", + "path": "", + "require_session": false, + "raw_body_only": false + }, + "response": [ + { + "disabled": false, + "name": "MyResponsePlugin", + "path": "", + "require_session": true, + "raw_body_only": false + } + ], + "driver": "grpc", + "id_extractor": { + "disabled": false, + "extract_from": "", + "extract_with": "", + "extractor_config": {} + } + } +} +``` + +In the above listing, the plugin driver parameter has been configured with a value of *grpc*. Two plugins are configured within the *custom_middleware* section: a *Pre Request* plugin and a *Response* plugin. + +The *Response* plugin is configured with *require_session* enabled, so that Tyk Gateway will send details for the authenticated key / user with the gRPC request. Note, this is not configured for *Pre Request* plugins that are triggered before authentication in the request lifecycle. + + +####### Tyk OAS API + +To quickly get started, a Tyk OAS API schema can be created by importing the infamous [pet store](https://petstore3.swagger.io/api/v3/openapi.json) OAS schema. Then the [findByStatus](https://petstore3.swagger.io/api/v3/pet/findByStatus?status=available) endpoint can be used for testing. + +The resulting Tyk OAS API Definition contains the OAS JSON schema with an *x-tyk-api-gateway* section appended, as listed below. gRPC plugins can be configured within the middleware section of the *x-tyk-api-gateway* that is appended at the end of the OAS schema: + +```yaml +"x-tyk-api-gateway": { + "info": { + "id": "6e2ae9b858734ea37eb772c666517f55", + "dbId": "65f457804773a600011af41d", + "orgId": "5e9d9544a1dcd60001d0ed20", + "name": "Swagger Petstore - OpenAPI 3.0 Custom Authentication", + "state": { + "active": true + } + }, + "upstream": { + "url": "https://petstore3.swagger.io/api/v3/" + }, + "server": { + "listenPath": { + "value": "/custom_auth", + "strip": true + }, + "authentication": { + "enabled": true, + "custom": { + "enabled": true, + "header": { + "enabled": false, + "name": "Authorization" + } + } + } + }, + "middleware": { + "global": { + "pluginConfig": { + "driver": "grpc" + } + }, + "cors": { + "enabled": false, + "maxAge": 24, + "allowedHeaders": [ + "Accept", + "Content-Type", + "Origin", + "X-Requested-With", + "Authorization" + ], + "allowedOrigins": [ + "*" + ], + "allowedMethods": [ + "GET", + "HEAD", + "POST" + ] + }, + "prePlugin": { + "api-management/plugins/overview#": [ + { + "enabled": true, + "functionName": "MyPreRequestPlugin", + "path": "" + } + ] + }, + "responsePlugin": { + "api-management/plugins/overview#": [ + { + "enabled": true, + "functionName": "MyResponsePlugin", + "path": "", + "requireSession": true + } + ] + } + } +} +``` + +In the above listing, the plugin driver parameter has been set to *grpc*. Two plugins are configured within the middleware section: a *Pre Request* plugin and a *Response* plugin. + +The *Response* plugin is configured with *requireSession* enabled, so that Tyk Gateway will send details for the authenticated key / user with the gRPC request. Note, this is not configurable for *Pre Request* plugins that are triggered before authentication in the request lifecycle. + +Tyk Gateway will forward details of an incoming request to the gRPC server, for each plugin. + + +#### Test API + +We have implemented and configured a gRPC server to integrate with Tyk Gateway. Furthermore, we have created an API that has been configured with two gRPC plugins: a *Pre Request* and *Response* plugin. + +When we issue a request to our API and observe the console output of our gRPC server we should see a JSON representation of the request headers etc. echoed in the terminal. + +Issue a request for your API in the terminal window. For example: + +```bash +curl -L http://.localhost:8080/grpc-http-bin +``` + +Observe the console output of your gRPC server. Tyk Gateway should have dispatched two requests to your gRPC server; a request for the *Pre Request* plugin and a request for the *Response* plugin. + +The gRPC server we implemented echoes a JSON representation of the request payload dispatched by Tyk Gateway. + +Note that this is a useful feature for learning how to develop gRPC plugins and understanding the structure of the request payload dispatched by Tyk Gateway to the gRPC server. However, in production environments care should be taken to avoid inadvertently exposing sensitive data such as secrets in the session. + + +#### Summary + +In this guide, we've delved into the integration of a Python gRPC server with Tyk Gateway. + +We have explained how to implement a Python gRPC server and equipped developers with the necessary tools, knowledge and capabilities to effectively utilize Tyk Gateway through gRPC services. + +The following essential groundwork has been covered: + +- Setting up tools, libraries and service definitions for the integration. +- Developing a basic gRPC server with functionality to echo the request payload, received from Tyk Gateway, in JSON format. +- Configuring Tyk Gateway for seamless communication with our gRPC server. + + +### Create a Request Transformation Plugin with Java + +This tutorial will guide you through the creation of a gRPC-based Java plugin for Tyk. +Our plugin will inject a header into the request before it gets proxied upstream. For additional information about gRPC, check the official documentation [here](https://grpc.io/docs/guides/index.html). + +The sample code that we'll use implements a request transformation plugin using Java and uses the proper gRPC bindings generated from our Protocol Buffers definition files. + +#### Requirements + +- Tyk Gateway: This can be installed using standard package management tools like Yum or APT, or from source code. See [here][1] for more installation options. +- The Tyk CLI utility, which is bundled with our RPM and DEB packages, and can be installed separately from [https://github.com/TykTechnologies/tyk-cli][2]. +- In Tyk 2.8 the Tyk CLI is part of the gateway binary, you can find more information by running "tyk help bundle". +- Gradle Build Tool: https://gradle.org/install/. +- gRPC tools: https://grpc.io/docs/quickstart/csharp.html#generate-grpc-code +- Java JDK 7 or higher. + +#### Create the Plugin + +##### Setting up the Java Project + +We will use the Gradle build tool to generate the initial files for our project: + +```bash +cd ~ +mkdir tyk-plugin +cd tyk-plugin +gradle init +``` + +We now have a `tyk-plugin` directory containing the basic skeleton of our application. + +Add the following to `build.gradle` + +```{.copyWrapper} +buildscript { + repositories { + jcenter() + } + dependencies { + classpath 'com.google.protobuf:protobuf-gradle-plugin:0.8.1' + } +} + +plugins { + id "com.google.protobuf" version "0.8.1" + id "java" + id "application" + id "idea" +} + +protobuf { + protoc { + artifact = "com.google.protobuf:protoc:3.3.0" + } + plugins { + grpc { + artifact = 'io.grpc:protoc-gen-grpc-java:1.5.0' + } + } + generateProtoTasks { + all()*.plugins { + grpc {} + } + } + generatedFilesBaseDir = "$projectDir/src/generated" +} + +sourceCompatibility = 1.8 +targetCompatibility = 1.8 + +mainClassName = "com.testorg.testplugin.PluginServer" + +repositories { + mavenCentral() +} + +dependencies { + compile 'io.grpc:grpc-all:1.5.0' +} + +idea { + module { + sourceDirs += file("${projectDir}/src/generated/main/java"); + sourceDirs += file("${projectDir}/src/generated/main/grpc"); + } +} +``` + +##### Create the Directory for the Server Class + +```bash +cd ~/tyk-plugin +mkdir -p src/main/java/com/testorg/testplugin +``` + +##### Install the gRPC Tools + +We need to download the Tyk Protocol Buffers definition files, these files contains the data structures used by Tyk. See [Data Structures]({{< ref "api-management/plugins/rich-plugins#rich-plugins-data-structures" >}}) for more information: + +```bash +cd ~/tyk-plugin +git clone https://github.com/TykTechnologies/tyk +mv tyk/coprocess/proto src/main/proto +``` + +##### Generate the Bindings + +To generate the Protocol Buffers bindings we use the Gradle build task: + +```bash +gradle build +``` + +If you need to customize any setting related to the bindings generation step, check the `build.gradle` file. + +##### Implement Server + +We need to implement two classes: one class will contain the request dispatcher logic and the actual middleware implementation. The other one will implement the gRPC server using our own dispatcher. + +From the `~/tyk-plugin/src/main/java/com/testorg/testplugin` directory, create a file named `PluginDispatcher.java` with the following code: + +```java +package com.testorg.testplugin; + +import coprocess.DispatcherGrpc; +import coprocess.CoprocessObject; + +public class PluginDispatcher extends DispatcherGrpc.DispatcherImplBase { + + @Override + public void dispatch(CoprocessObject.Object request, + io.grpc.stub.StreamObserver responseObserver) { + CoprocessObject.Object modifiedRequest = null; + + switch (request.getHookName()) { + case "MyPreMiddleware": + modifiedRequest = MyPreHook(request); + default: + // Do nothing, the hook name isn't implemented! + } + + // Return the modified request (if the transformation was done): + if (modifiedRequest != null) { + responseObserver.onNext(modifiedRequest); + }; + + responseObserver.onCompleted(); + } + + CoprocessObject.Object MyPreHook(CoprocessObject.Object request) { + CoprocessObject.Object.Builder builder = request.toBuilder(); + builder.getRequestBuilder().putSetHeaders("customheader", "customvalue"); + return builder.build(); + } +} +``` + +In the same directory, create a file named `PluginServer.java` with the following code. This is the server implementation: + +```java +package com.testorg.testplugin; + +import coprocess.DispatcherGrpc; + +import io.grpc.Server; +import io.grpc.ServerBuilder; +import io.grpc.stub.StreamObserver; +import java.io.IOException; +import java.util.logging.Level; +import java.util.logging.Logger; + +public class PluginServer { + + private static final Logger logger = Logger.getLogger(PluginServer.class.getName()); + static Server server; + static int port = 5555; + + public static void main(String[] args) throws IOException, InterruptedException { + System.out.println("Initializing gRPC server."); + + // Our dispatcher is instantiated and attached to the server: + server = ServerBuilder.forPort(port) + .addService(new PluginDispatcher()) + .build() + .start(); + + blockUntilShutdown(); + + } + + static void blockUntilShutdown() throws InterruptedException { + if (server != null) { + server.awaitTermination(); + } + } +} +``` + +To run the gRPC server we can use the following command: + +```bash +cd ~/tyk-plugin +gradle runServer +``` + +The gRPC server will listen on port 5555 (as defined in `Server.java`). In the next steps we'll setup the plugin bundle and modify Tyk to connect to our gRPC server. + + +#### Bundle the Plugin + +We need to create a manifest file within the `tyk-plugin` directory. This file contains information about our plugin and how we expect it to interact with the API that will load it. This file should be named `manifest.json` and needs to contain the following: + +```json +{ + "custom_middleware": { + "driver": "grpc", + "pre": [{ + "name": "MyPreMiddleware" + }] + } +} +``` + +- The `custom_middleware` block contains the middleware settings like the plugin driver we want to use (`driver`) and the hooks that our plugin will expose. We use the `pre` hook for this tutorial. For other hooks see [here]({{< ref "api-management/plugins/rich-plugins#coprocess-dispatcher---hooks" >}}). +- The `name` field references the name of the function that we implemented in our plugin code - `MyPreMiddleware`. This will be handled by our dispatcher gRPC method in `PluginServer.java`. + +To bundle our plugin run the following command in the `tyk-plugin` directory. Check your tyk-cli install path first: + +```bash +/opt/tyk-gateway/utils/tyk-cli bundle build -y +``` + +For Tyk 2.8 use: +```bash +/opt/tyk-gateway/bin/tyk bundle build -y +``` + +A plugin bundle is a packaged version of the plugin. It may also contain a cryptographic signature of its contents. The `-y` flag tells the Tyk CLI tool to skip the signing process in order to simplify the flow of this tutorial. + +For more information on the Tyk CLI tool, see [here]({{< ref "api-management/plugins/overview#plugin-bundles" >}}). + +You should now have a `bundle.zip` file in the `tyk-plugin` directory. + +#### Publish the Plugin + +To publish the plugin, copy or upload `bundle.zip` to a local web server like Nginx, or Apache or storage like Amazon S3. For this tutorial we'll assume you have a web server listening on `localhost` and accessible through `http://localhost`. + +{{< include "grpc-include" >}} + +#### What's Next? + +In this tutorial we learned how Tyk gRPC plugins work. For a production-level setup we suggest the following: + +- Configure an appropriate web server and path to serve your plugin bundles. + +[1]: https://tyk.io/docs/get-started/with-tyk-on-premise/installation/ +[2]: https://github.com/TykTechnologies/tyk-cli +[3]: /img/dashboard/system-management/api_settings.png +[4]: /img/dashboard/system-management/plugin_options.png + +### Create Custom Authentication Plugin with .NET + +This tutorial will guide you through the creation of a custom authentication plugin for Tyk with a gRPC based plugin with .NET and C#. For additional information check the official gRPC [documentation](https://grpc.io/docs/guides/index.html). + +The sample code that we’ll use implements a very simple authentication layer using .NET and the proper gRPC bindings generated from our Protocol Buffers definition files. + +{{< img src="/img/diagrams/diagram_docs_gRPC-plugins_why-use-it-for-plugins@2x.png" alt="Using gRPC for plugins" >}} + +#### Requirements + +- Tyk Gateway: This can be installed using standard package management tools like Yum or APT, or from source code. See [here][1] for more installation options. +- The Tyk CLI utility, which is bundled with our RPM and DEB packages, and can be installed separately from [https://github.com/TykTechnologies/tyk-cli][2] +- In Tyk 2.8 the Tyk CLI is part of the gateway binary, you can find more information by running "tyk help bundle". +- .NET Core for your OS: https://www.microsoft.com/net/core +- gRPC tools: https://grpc.io/docs/quickstart/csharp.html#generate-grpc-code + +#### Create the Plugin + +##### Create .NET Project + +We use the .NET CLI tool to generate the initial files for our project: + +```bash +cd ~ +dotnet new console -o tyk-plugin +``` + +We now have a `tyk-plugin` directory containing the basic skeleton of a .NET application. + +From the `tyk-plugin` directory we need to install a few packages that the gRPC server requires: + +```bash +dotnet add package Grpc --version 1.6.0 +dotnet add package System.Threading.ThreadPool --version 4.3.0 +dotnet add package Google.Protobuf --version 3.4.0 +``` + +- The `Grpc` package provides base code for our server implementation. +- The `ThreadPool` package is used by `Grpc`. +- The `Protobuf` package will be used by our gRPC bindings. + +##### Install the gRPC Tools + +We need to install the gRPC tools to generate the bindings. We recommended you follow the official guide here: https://grpc.io/docs/quickstart/csharp.html#generate-grpc-code. + +Run the following Commands (both MacOS and Linux): + +```bash +cd ~/tyk-plugin +temp_dir=packages/Grpc.Tools.1.6.x/tmp +curl_url=https://www.nuget.org/api/v2/package/Grpc.Tools/ +mkdir -p $temp_dir && cd $temp_dir && curl -sL $curl_url > tmp.zip; unzip tmp.zip && cd .. && cp -r tmp/tools . && rm -rf tmp && cd ../.. +chmod -Rf +x packages/Grpc.Tools.1.6.x/tools/ +``` + +Then run the following, depending on your OS: + +**MacOS (x64)** + +```bash +export GRPC_TOOLS=packages/Grpc.Tools.1.6.x/tools/macosx_x64 +``` + +**Linux (x64)** + +```bash +export GRPC_TOOLS=packages/Grpc.Tools.1.6.x/tools/linux_x64 +``` + +The `GRPC_TOOLS` environment variable will point to the appropriate GrpcTools path that matches our operating system and architecture. The last step is to export a variable for the `protoc` program; this is the main program used to generate bindings: + +```bash +export GRPC_PROTOC=$GRPC_TOOLS/protoc +``` + +Now that we can safely run `protoc`, we can download the Tyk Protocol Buffers definition files. These files contain the data structures used by Tyk. See [Data Structures]({{< ref "api-management/plugins/rich-plugins#rich-plugins-data-structures" >}}) for more information: + +```bash +cd ~/tyk-plugin +git clone https://github.com/TykTechnologies/tyk +``` + +##### Generate the bindings + +To generate the bindings, we create an empty directory and run the `protoc` tool using the environment variable that was set before: + +```bash +mkdir Coprocess +$GRPC_PROTOC -I=tyk/coprocess/proto --csharp_out=Coprocess --grpc_out=Coprocess --plugin=protoc-gen-grpc=$GRPC_TOOLS/grpc_csharp_plugin tyk/coprocess/proto/*.proto +``` + +Run the following command to check the binding directory: + +```bash +ls Coprocess +``` + +The output will look like this: + +``` +CoprocessCommon.cs CoprocessObject.cs CoprocessReturnOverrides.cs +CoprocessMiniRequestObject.cs CoprocessObjectGrpc.cs CoprocessSessionState.cs +``` + +##### Implement Server + +Create a file called `Server.cs`. + +Add the following code to `Server.cs`. + +```c# +using System; +using System.Threading.Tasks; +using Grpc.Core; + +using Coprocess; + +class DispatcherImpl : Dispatcher.DispatcherBase +{ + public DispatcherImpl() + { + Console.WriteLine("Instantiating DispatcherImpl"); + } + + + // The Dispatch method will be called by Tyk for every configured hook, we'll implement a very simple dispatcher here: + public override Task Dispatch(Coprocess.Object thisObject, ServerCallContext context) + { + // thisObject is the request object: + Console.WriteLine("Receiving object: " + thisObject.ToString()); + + // hook contains the hook name, this will be defined in our plugin bundle and the implementation will be a method in this class (DispatcherImpl), we'll look it up: + var hook = this.GetType().GetMethod(thisObject.HookName); + + // If hook is null then a handler method for this hook isn't implemented, we'll log this anyway: + if (hook == null) + { + Console.WriteLine("Hook name: " + thisObject.HookName + " (not implemented!)"); + // We return the unmodified request object, so that Tyk can proxy this in the normal way. + return Task.FromResult(thisObject); + }; + + // If there's a handler method, let's log it and proceed with our dispatch work: + Console.WriteLine("Hook name: " + thisObject.HookName + " (implemented)"); + + // This will dynamically invoke our hook method, and cast the returned object to the required Protocol Buffers data structure: + var output = hook.Invoke(this, new object[] { thisObject, context }); + return (Task)output; + } + + // MyPreMiddleware implements a PRE hook, it will be called before the request is proxied upstream and before the authentication step: + public Task MyPreMiddleware(Coprocess.Object thisObject, ServerCallContext context) + { + Console.WriteLine("Calling MyPreMiddleware."); + // We'll inject a header in this request: + thisObject.Request.SetHeaders["my-header"] = "my-value"; + return Task.FromResult(thisObject); + } + + // MyAuthCheck implements a custom authentication mechanism, it will initialize a session object if the token matches a certain value: + public Task MyAuthCheck(Coprocess.Object thisObject, ServerCallContext context) + { + // Request.Headers contains all the request headers, we retrieve the authorization token: + var token = thisObject.Request.Headers["Authorization"]; + Console.WriteLine("Calling MyAuthCheck with token = " + token); + + // We initialize a session object if the token matches "abc123": + if (token == "abc123") + { + Console.WriteLine("Successful auth!"); + var session = new Coprocess.SessionState(); + session.Rate = 1000; + session.Per = 10; + session.QuotaMax = 60; + session.QuotaRenews = 1479033599; + session.QuotaRemaining = 0; + session.QuotaRenewalRate = 120; + session.Expires = 1479033599; + + session.LastUpdated = 1478033599.ToString(); + + thisObject.Metadata["token"] = token; + thisObject.Session = session; + return Task.FromResult(thisObject); + + } + + // If the token isn't "abc123", we return the request object in the original state, without a session object, Tyk will reject this request: + Console.WriteLine("Rejecting auth!"); + return Task.FromResult(thisObject); + } +} +``` + +Create a file called `Program.cs` to instantiate our dispatcher implementation and start a gRPC server. + +Add the following code to `Program.cs`. + +```bash +using System; +using Grpc.Core; + +namespace tyk_plugin +{ + class Program + { + + // Port to attach the gRPC server to: + const int Port = 5555; + + static void Main(string[] args) + { + // We initialize a Grpc.Core.Server and attach our dispatcher implementation to it: + Server server = new Server + { + Services = { Coprocess.Dispatcher.BindService(new DispatcherImpl()) }, + Ports = { new ServerPort("localhost", Port, ServerCredentials.Insecure) } + }; + server.Start(); + + Console.WriteLine("gRPC server listening on " + Port); + Console.WriteLine("Press any key to stop the server..."); + Console.ReadKey(); + + server.ShutdownAsync().Wait(); + + } + } +} +``` + +To run the gRPC server use the following command from the plugin directory: + +```bash +dotnet run +``` + +The gRPC server will listen on port 5555 (as defined in `Program.cs`). In the next steps we'll setup the plugin bundle and modify Tyk to connect to our gRPC server. + +#### Bundle the Plugin + +We need to create a manifest file within the `tyk-plugin` directory. This file contains information about our plugin and how we expect it to interact with the API that will load it. This file should be named `manifest.json` and needs to contain the following: + +```json +{ + "custom_middleware": { + "driver": "grpc", + "auth_check": { + "name": "MyAuthMiddleware", + "path": "", + "raw_body_only": false, + "require_session": false + } + } +} +``` + +- The `custom_middleware` block contains the middleware settings like the plugin driver we want to use (`driver`) and the hooks that our plugin will expose. We use the `auth_check` hook for this tutorial. For other hooks see [here]({{< ref "api-management/plugins/rich-plugins#coprocess-dispatcher---hooks" >}}). +- The `name` field references the name of the function that we implement in our plugin code - `MyAuthMiddleware`. This will be handled by our dispatcher gRPC method (implemented in `Server.cs`). +- The `path` field is the path to the middleware component. +- The `raw_body_only` field +- The `require_session` field, if set to `true` gives you access to the session object. It will be supplied as a session variable to your middleware processor function + + +To bundle our plugin run the following command in the `tyk-plugin` directory. Check your tyk-cli install path first: + +```bash +/opt/tyk-gateway/utils/tyk-cli bundle build -y +``` + +From Tyk v2.8 upwards you can use: +```bash +/opt/tyk-gateway/bin/tyk bundle build -y +``` + +A plugin bundle is a packaged version of the plugin. It may also contain a cryptographic signature of its contents. The `-y` flag tells the Tyk CLI tool to skip the signing process in order to simplify the flow of this tutorial. + +For more information on the Tyk CLI tool, see [here]({{< ref "api-management/plugins/overview#plugin-bundles" >}}). + +You should now have a `bundle.zip` file in the `tyk-plugin` directory. + +#### Publish the Plugin + +To publish the plugin, copy or upload `bundle.zip` to a local web server like Nginx, or Apache or storage like Amazon S3. For this tutorial we'll assume you have a web server listening on `localhost` and accessible through `http://localhost`. + +{{< include "grpc-include" >}} + +#### What's Next? + +In this tutorial we learned how Tyk gRPC plugins work. For a production-level setup we suggest the following: + +- Configure an appropriate web server and path to serve your plugin bundles. +- See the following [GitHub repo](https://github.com/TykTechnologies/tyk-plugin-demo-dotnet) for a gRPC based .NET plugin that incorporates authentication based on Microsoft SQL Server. + +[1]: https://tyk.io/docs/get-started/with-tyk-on-premise/installation/ +[2]: https://github.com/TykTechnologies/tyk-cli +[3]: /img/dashboard/system-management/plugin_options.png +[4]: /img/dashboard/system-management/plugin_auth_mode.png + +### Create Custom Authentication Plugin with NodeJS + +This tutorial will guide you through the creation of a custom authentication plugin for Tyk with a gRPC based plugin written in NodeJS. For additional information about gRPC, check the official documentation [here](https://grpc.io/docs/guides/index.html). + +The sample code that we'll use implements a very simple authentication layer using NodeJS and the proper gRPC bindings generated from our Protocol Buffers definition files. + +{{< img src="/img/dashboard/system-management/custom_grpc_authentication.png" alt="gRPC Auth Diagram" >}} + +#### Requirements + +- Tyk Gateway: This can be installed using standard package management tools like Yum or APT, or from source code. See [here](https://tyk.io/docs/get-started/with-tyk-on-premise/installation/) for more installation options. +- The Tyk CLI utility, which is bundled with our RPM and DEB packages, and can be installed separately from [https://github.com/TykTechnologies/tyk-cli](https://github.com/TykTechnologies/tyk-cli) +- In Tyk 2.8 and upwards the Tyk CLI is part of the gateway binary, you can find more information by running "tyk help bundle". +- NodeJS v6.x.x [https://nodejs.org/en/download/](https://nodejs.org/en/download/) + +#### Create the Plugin + +##### Create NodeJS Project + +We will use the NPM tool to initialize our project, follow the steps provided by the `init` command: + +```bash +cd ~ +mkdir tyk-plugin +cd tyk-plugin +npm init +``` + +Now we'll add the gRPC package for this project: + +```bash +npm install --save grpc +``` + +##### Install gRPC Tools + +Typically to use gRPC and Protocol Buffers you need to use a code generator and generate bindings for the target language that you're using. For this tutorial we'll skip this step and use the dynamic loader that's provided by the NodeJS gRPC library. This mechanism allows a program to load Protocol Buffers definitions directly from `.proto` files. See [this section](https://grpc.io/docs/tutorials/basic/node.html#loading-service-descriptors-from-proto-files) in the gRPC documentation for more details. + +To fetch the required `.proto` files, you may use an official repository where we keep the Tyk Protocol Buffers definition files: + +```bash +cd ~/tyk-plugin +git clone https://github.com/TykTechnologies/tyk +``` + +##### Implement Server + +Now we're ready to implement our gRPC server, create a file called `main.js` in the project's directory + +Add the following code to `main.js`. + +```nodejs +const grpc = require('grpc'), + resolve = require('path').resolve + +const tyk = grpc.load({ + file: 'coprocess_object.proto', + root: resolve(__dirname, 'tyk/coprocess/proto') +}).coprocess + +const listenAddr = '127.0.0.1:5555', + authHeader = 'Authorization' + validToken = '71f6ac3385ce284152a64208521c592b' + +// The dispatch function is called for every hook: +const dispatch = (call, callback) => { + var obj = call.request + // We dispatch the request based on the hook name, we pass obj.request which is the coprocess.Object: + switch (obj.hook_name) { + case 'MyPreMiddleware': + preMiddleware(obj, callback) + break + case 'MyAuthMiddleware': + authMiddleware(obj, callback) + break + default: + callback(null, obj) + break + } +} + +const preMiddleware = (obj, callback) => { + var req = obj.request + + // req is the coprocess.MiniRequestObject, we inject a header using the "set_headers" field: + req.set_headers = { + 'mycustomheader': 'mycustomvalue' + } + + // Use this callback to finish the operation, sending back the modified object: + callback(null, obj) +} + +const authMiddleware = (obj, callback) => { + var req = obj.request + + // We take the value from the "Authorization" header: + var token = req.headers[authHeader] + + // The token should be attached to the object metadata, this is used internally for key management: + obj.metadata = { + token: token + } + + // If the request token doesn't match the "validToken" constant we return the call: + if (token != validToken) { + callback(null, obj) + return + } + + // At this point the token is valid and a session state object is initialized and attached to the coprocess.Object: + var session = new tyk.SessionState() + session.id_extractor_deadline = Date.now() + 100000000000 + obj.session = session + callback(null, obj) +} + +main = function() { + server = new grpc.Server() + server.addService(tyk.Dispatcher.service, { + dispatch: dispatch + }) + server.bind(listenAddr, grpc.ServerCredentials.createInsecure()) + server.start() +} + +main() +``` + + +To run the gRPC server run: + +```bash +node main.js +``` + +The gRPC server will listen on port `5555` (see the `listenAddr` constant). In the next steps we'll setup the plugin bundle and modify Tyk to connect to our gRPC server. + + +#### Bundle the Plugin + +We need to create a manifest file within the `tyk-plugin` directory. This file contains information about our plugin and how we expect it to interact with the API that will load it. This file should be named `manifest.json` and needs to contain the following: + +```json +{ + "custom_middleware": { + "driver": "grpc", + "auth_check": { + "name": "MyAuthMiddleware", + "path": "", + "raw_body_only": false, + "require_session": false + } + } +} +``` + +- The `custom_middleware` block contains the middleware settings like the plugin driver we want to use (`driver`) and the hooks that our plugin will expose. We use the `auth_check` hook for this tutorial. For other hooks see [here]({{< ref "api-management/plugins/rich-plugins#coprocess-dispatcher---hooks" >}}). +- The `name` field references the name of the function that we implement in our plugin code - `MyAuthMiddleware`. The implemented dispatcher uses a switch statement to handle this hook, and calls the `authMiddleware` function in `main.js`. +- The `path` field is the path to the middleware component. +- The `raw_body_only` field +- The `require_session` field, if set to `true` gives you access to the session object. It will be supplied as a session variable to your middleware processor function + +To bundle our plugin run the following command in the `tyk-plugin` directory. Check your tyk-cli install path first: + +```bash +/opt/tyk-gateway/utils/tyk-cli bundle build -y +``` + +For Tyk 2.8 use: +```bash +/opt/tyk-gateway/bin/tyk bundle build -y +``` + +A plugin bundle is a packaged version of the plugin. It may also contain a cryptographic signature of its contents. The `-y` flag tells the Tyk CLI tool to skip the signing process in order to simplify the flow of this tutorial. + +For more information on the Tyk CLI tool, see [here]({{< ref "api-management/plugins/overview#plugin-bundles" >}}). + +You should now have a `bundle.zip` file in the `tyk-plugin` directory. + +#### Publish the Plugin + +To publish the plugin, copy or upload `bundle.zip` to a local web server like Nginx, Apache or storage like Amazon S3. For this tutorial we'll assume you have a web server listening on `localhost` and accessible through `http://localhost`. + +{{< include "grpc-include" >}} + + +#### What's Next? + +In this tutorial we learned how Tyk gRPC plugins work. For a production-level setup we suggest the following: + +- Configure an appropriate web server and path to serve your plugin bundles. + +[1]: https://tyk.io/docs/get-started/with-tyk-on-premise/installation/ +[2]: https://github.com/TykTechnologies/tyk-cli +[3]: /img/dashboard/system-management/plugin_options.png +[4]: /img/dashboard/system-management/plugin_auth_mode.png + +### Create Custom Authentication Plugin With Python + +In the realm of API security, HMAC-signed authentication serves as a foundational concept. In this developer-focused blog post, we'll use HMAC-signed authentication as the basis for learning how to write gRPC custom authentication plugins with Tyk Gateway. Why learn how to write Custom Authentication Plugins? + +- **Foundational knowledge**: Writing custom authentication plugins provides foundational knowledge of Tyk's extensibility and customization capabilities. +- **Practical experience**: Gain hands-on experience in implementing custom authentication logic tailored to specific use cases, starting with HMAC-signed authentication. +- **Enhanced control**: Exercise greater control over authentication flows and response handling, empowering developers to implement advanced authentication mechanisms beyond built-in features. + +While Tyk Gateway offers built-in support for HMAC-signed authentication, this tutorial serves as a practical guide for developers looking to extend Tyk's capabilities through custom authentication plugins. It extends the gRPC server that we developed in our [getting started guide]({{< ref "api-management/plugins/rich-plugins#using-python" >}}). + +We will develop a basic gRPC server that implements the Tyk Dispatcher service with a custom authentication plugin to handle authentication keys, signed using the HMAC SHA512 algorithm. Subsequently, you will be able to make a request to your API with a HMAC signed authentication key in the *Authorization* header. Tyk Gateway will intercept the request and forward it to your Python gRPC server for HMAC signature and token verification. + +Our plugin will only verify the key against an expected value. In a production environment it will be necessary to verify the key against Redis storage. + +Before we continue ensure that you have: + +- Read and completed our getting started guide that explains how to implement a basic Python gRPC server to echo the request payload received from Tyk Gateway. This tutorial extends the source code of the tyk_async_server.py file to implement a custom authentication plugin for a HMAC signed authentication key. +- Read our HMAC signatures documentation for an explanation of HMAC signed authentication with Tyk Gateway. A brief summary is given in the HMAC Signed Authentication section below. + + +#### HMAC Signed Authentication + +Before diving in further, we will give a brief overview of HMAC signed authentication using our custom authentication plugin. + +- **Client request**: The journey begins with a client requesting access to a protected resource on the Tyk API. +- **HMAC signing**: Before dispatching the request, the client computes an HMAC signature using a secret key and request date, ensuring the payload's integrity. +- **Authorization header**: The HMAC signature, along with essential metadata such as the API key and HMAC algorithm, is embedded within the Authorization header. +- **Tyk Gateway verification**: Upon receipt, Tyk Gateway forwards the request to our gRPC server to execute the custom authentication plugin. This will validate the HMAC signature, ensuring the request's authenticity before proceeding with further processing. + +Requests should be made to an API that uses our custom authentication plugin as follows. A HMAC signed key should be included in the *Authorization* header and a date/time string in the *Date* header. An example request is shown in the curl command below: + +```bash +curl -v -H 'Date: Fri, 03 May 2024 12:00:42 GMT' \ +-H 'Authorization: Signature keyId="eyJvcmciOiI1ZTlkOTU0NGExZGNkNjAwMDFkMGVkMjAiLCJpZCI6ImdycGNfaG1hY19rZXkiLCJoIjoibXVybXVyNjQifQ==", \ +algorithm="hmac-sha512",signature="9kwBK%2FyrjbSHJDI7INAhBmhHLTHRDkIe2uRWHEP8bgQFQvfXRksm6t2MHeLUyk9oosWDZyC17AbGeP8EFqrp%2BA%3D%3D"' \ +http://localhost:8080/grpc-custom-auth/get +``` + +From the above example, it should be noted that: + +- The *Date* header contains a date string formatted as follows: *Fri, 03 May 2024 11:06:00 GMT*. +- The *Authorization* header is formatted as *Signature keyId=””, algorithm=””, signature=””* where: + + - **keyId** is a Tyk authentication key. + - **algorithm** is the HMAC algorithm used to sign the signature, *hmac-sha512* or *hmac-sha256*. + - **signature** is the HAMC signature calculated with the date string from the *Date* header, signed with a base64 encoded secret value, using the specified HMAC algorithm. The HMAC signature is then encoded as base64. + +#### Prerequisites + +Firstly, we need to create the following: + +- An API configured to use a custom authentication plugin. +- A HMAC enabled key with a configured secret for signing. + +This will enable us to issue a request to test that Tyk Gateway integrates with our custom authentication plugin on the gRPC server. + +###### Create API + +We will create an API served by Tyk Gateway, that will forward requests upstream to https://httpbin.org/. + +The API will have the following parameters configured: + +- **Listen path**: Tyk Gateway will listen to API requests on */grpc-custom-auth/* and will strip the listen path for upstream requests. +- **Target URL**: The target URL will be configured to send requests to *http://httpbin/*. +- **Authentication Mode**: The authentication mode will be configured for custom authentication. This is used to trigger CoProcess (gRPC), Python or JSVM plugins to handle custom authentication. + +You can use the following Tyk Classic API definition to get you started, replacing the *org_id* with the ID of your organization. + +```json +{ + "api_definition": { + "id": "662facb2f03e750001a03500", + "api_id": "6c56dd4d3ad942a94474df6097df67ed", + "org_id": "5e9d9544a1dcd60001d0ed20", + "name": "Python gRPC Custom Auth", + "enable_coprocess_auth": true, + "auth": { + "auth_header_name": "Authorization" + }, + "proxy": { + "preserve_host_header": false, + "listen_path": "/grpc-custom-auth/", + "disable_strip_slash": true, + "strip_listen_path": true, + "target_url": "http://httpbin/" + }, + "version_data": { + "not_versioned": false, + "versions": { + "Default": { + "name": "Default", + "expires": "", + "use_extended_paths": true, + "extended_paths": { + "ignored": [], + "white_list": [], + "black_list": [] + } + } + }, + "default_version": "Default" + }, + "active": true + } +} +``` + +The Tyk API definition above can be imported via Tyk Dashboard. Alternatively, if using Tyk Gateway OSS, a POST request can be made to the *api/apis* endpoint of Tyk Gateway. Consult the [Tyk Gateway Open API Specification documentation]({{< ref "tyk-gateway-api" >}}) for usage. + +An illustrative example using *curl* is given below. Please note that you will need to: + +- Update the location to use the protocol scheme, host and port suitable for your environment. +- Replace the value in the *x-tyk-authorization* header with the secret value in your *tyk.conf* file. +- Replace the *org_id* with the ID of your organization. + +```bash +curl -v \ + --header 'Content-Type: application/json' \ + --header 'x-tyk-authorization: your Gateway admin secret' \ + --location http://localhost:8080/tyk/apis/ \ + --data '{\ + "api_definition": {\ + "id": "662facb2f03e750001a03502",\ + "api_id": "6c56dd4d3ad942a94474df6097df67ef",\ + "org_id": "5e9d9544a1dcd60001d0ed20",\ + "name": "Python gRPC Custom Auth",\ + "enable_coprocess_auth": true,\ + "auth": {\ + "auth_header_name": "Authorization"\ + },\ + "proxy": {\ + "preserve_host_header": false,\ + "listen_path": "/grpc-custom-auth-error/",\ + "disable_strip_slash": true,\ + "strip_listen_path": true,\ + "target_url": "http://httpbin/"\ + },\ + "version_data": {\ + "not_versioned": false,\ + "versions": {\ + "Default": {\ + "name": "Default",\ + "expires": "",\ + "use_extended_paths": true,\ + "extended_paths": {\ + "ignored": [],\ + "white_list": [],\ + "black_list": []\ + }\ + }\ + },\ + "default_version": "Default"\ + },\ + "active": true\ + }\ + }' +``` + +A response similar to that given below will be returned by Tyk Gateway: + +```bash +{ + "key": "f97b748fde734b099001ca15f0346dfe", + "status": "ok", + "action": "added" +} +``` + +###### Create HMAC Key + +We will create an key configured to use HMAC signing, with a secret of *secret*. The key will configured to have access to our test API. + +You can use the following configuration below, replacing the value of the *org_id* with the ID of your organization. + +```bash +{ + "quota_max": 1000, + "quota_renews": 1596929526, + "quota_remaining": 1000, + "quota_reset": 1596843126, + "quota_used": 0, + "org_id": "5e9d9544a1dcd60001d0ed20", + "access_rights": { + "662facb2f03e750001a03500": { + "api_id": "662facb2f03e750001a03500", + "api_name": "Python gRPC Custom Auth", + "versions": ["Default"], + "allowed_urls": [], + "limit": null, + "quota_max": 1000, + "quota_renews": 1596929526, + "quota_remaining": 1000, + "quota_reset": 1596843126, + "quota_used": 0, + "per": 1, + "expires": -1 + } + }, + "enable_detailed_recording": true, + "hmac_enabled": true, + "hmac_string": "secret", + "meta_data": {} +} +``` + +You can use Tyk Gateway’s API to create the key by issuing a POST request to the *tyk/keys* endpoint. Consult the [Tyk Gateway Open API Specification documentation]({{< ref "tyk-gateway-api" >}}) for usage. + +An illustrative example using *curl* is given below. Please note that you will need to: + +- Update the location to use the protocol scheme, host and port suitable for your environment. +- Replace the value in the *x-tyk-authorization* header with the secret value in your *tyk.conf* file. + +Replace the *org_id* with the ID of your organization. + +```bash +curl --location 'http://localhost:8080/tyk/keys/grpc_hmac_key' \ +--header 'x-tyk-authorization: your Gateay admin secret' \ +--header 'Content-Type: application/json' \ +--data '{\ + "alias": "grpc_hmac_key",\ + "quota_max": 1000,\ + "quota_renews": 1596929526,\ + "quota_remaining": 1000,\ + "quota_reset": 1596843126,\ + "quota_used": 0,\ + "org_id": "5e9d9544a1dcd60001d0ed20",\ + "access_rights": {\ + "662facb2f03e750001a03500": {\ + "api_id": "662facb2f03e750001a03500",\ + "api_name": "python-grpc-custom-auth",\ + "versions": ["Default"],\ + "allowed_urls": [],\ + "limit": null,\ + "quota_max": 1000,\ + "quota_renews": 1596929526,\ + "quota_remaining": 1000,\ + "quota_reset": 1596843126,\ + "quota_used": 0,\ + "per": 1,\ + "expires": -1\ + }\ + },\ + "enable_detailed_recording": true,\ + "hmac_enabled": true,\ + "hmac_string": "secret",\ + "meta_data": {}\ +}\ +' +``` + +A response similar to that given below should be returned by Tyk Gateway: + +```json +{ + "key": "eyJvcmciOiI1ZTlkOTU0NGExZGNkNjAwMDFkMGVkMjAiLCJpZCI6ImdycGNfaG1hY19rZXkiLCJoIjoibXVybXVyNjQifQ==", + "status": "ok", + "action": "added", + "key_hash": "a72fcdc09caa86b5" +} +``` + +{{< note success>}} + +**Note** + +Make a note of the key ID given in the response, since we will need this to test our API. +{{< /note >}} + +#### Implement Plugin + +Our custom authentication plugin will perform the following tasks: + +- Extract the *Authorization* and *Date* headers from the request object. +- Parse the *Authorization* header to extract the *keyId*, *algorithm* and *signature* attributes. +- Compute the HMAC signature using the specific algorithm and date included in the header. +- Verify that the computed HMAC signature matches the signature included in the *Authorization* header. A 401 error response will be returned if verification fails. Our plugin will only verify the key against an expected value. In a production environment it will be necessary to verify the key against Redis storage. +- Verify that the *keyId* matches an expected value (VALID_TOKEN). A 401 error response will be returned to Tyk Gateway if verification fails. +- If verification of the signature and key passes then update the session with HMAC enabled and set the HMAC secret. Furthermore, add the key to the *Object* metadata. + +Return the request *Object* containing the updated session back to Tyk Gateway. When developing custom authentication plugins it is the responsibility of the developer to update the session state with the token, in addition to setting the appropriate response status code and error message when authentication fails. + +##### Import Python Modules + +Ensure that the following Python modules are imported at the top of your *tyk_async_server.py* file: + +```python +import asyncio +import base64 +import hashlib +import hmac +import json +import re +import signal +import logging +import urllib.parse + +import grpc +from google.protobuf.json_format import MessageToJson +from grpc_reflection.v1alpha import reflection +import coprocess_object_pb2_grpc +import coprocess_object_pb2 +from coprocess_common_pb2 import HookType +from coprocess_session_state_pb2 import SessionState +``` + +##### Add Constants + +Add the following constants to the top of the *tyk_async_server.py* file, after the import statements: + +```bash +SECRET = "c2VjcmV0" +VALID_TOKEN = "eyJvcmciOiI1ZTlkOTU0NGExZGNkNjAwMDFkMGVkMjAiLCJpZCI6ImdycGNfaG1hY19rZXkiLCJoIjoibXVybXVyNjQifQ==" +``` + +- **SECRET** is a base64 representation of the secret used for HMAC signing. +- **VALID_TOKEN** is the key ID that we will authenticate against. + +The values listed above are designed to align with the examples provided in the *Prerequisites* section, particularly those related to HMAC key generation. If you've made adjustments to the HMAC secret or you've modified the key alias referred to in the endpoint path (for instance, *grpc_hmac_key*), you'll need to update these constants accordingly. + +##### Extract headers + +Add the following function to your *tyk_async_server.py* file to extract a dictionary of the key value pairs from the *Authorization* header. We will use a regular expression to extract the key value pairs. + +```python +def parse_auth_header(auth_header: str) -> dict[str,str]: + pattern = r'(\w+)\s*=\s*"([^"]+)"' + + matches = re.findall(pattern, auth_header) + + parsed_data = dict(matches) + + return parsed_data +``` + +##### Compute HMAC Signature + +Add the following function to your *tyk_async_server.py* to compute the HMAC signature. + +```python +def generate_hmac_signature(algorithm: str, date_string: str, secret_key: str) -> str: + + if algorithm == "hmac-sha256": + hash_algorithm = hashlib.sha256 + elif algorithm == "hmac-sha512": + hash_algorithm = hashlib.sha512 + else: + raise ValueError("Unsupported hash algorithm") + + base_string = f"date: {date_string}" + + logging.info(f"generating signature from: {base_string}") + hmac_signature = hmac.new(secret_key.encode(), base_string.encode(), hash_algorithm) + + return base64.b64encode(hmac_signature.digest()).decode() +``` + +Our function accepts three parameters: + +- **algorithm** is the HMAC algorithm to use for signing. We will use HMAC SHA256 or HMAC SHA512 in our custom authentication plugin +- **date_string** is the date extracted from the date header in the request sent by Tyk Gateway. +- **secret_key** is the value of the secret used for signing. + +The function computes and returns the HMAC signature for a string formatted as *date: date_string*, where *date_string* corresponds to the value of the *date_string* parameter. The signature is computed using the secret value given in the *secret_key* parameter and the HMAC algorithm given in the *algorithm* parameter. A *ValueError* is raised if the hash algorithm is unrecognized. + +We use the following Python modules in our implementation: + +- hmac Python module to compute the HMAC signature. +- base64 Python module to encode the result. + +##### Verify HMAC Signature + +Add the following function to your *tyk_async_server.py* file to verify the HMAC signature provided by the client: + +```python +def verify_hmac_signature(algorithm: str, signature: str, source_string) -> bool: + + expected_signature = generate_hmac_signature(algorithm, source_string, SECRET) + received_signature = urllib.parse.unquote(signature) + + if expected_signature != received_signature: + error = f"Signatures did not match\nreceived: {received_signature}\nexpected: {expected_signature}" + logging.error(error) + else: + logging.info("Signatures matched!") + + return expected_signature == received_signature +``` + +Our function accepts three parameters: + +- **algorithm** is the HMAC algorithm to use for signing. We will use hmac-sha256 or hmac-sha512 in our custom authentication plugin. +- **signature** is the signature string extracted from the *Authorization* header. +- **source_string** is the date extracted from the date header in the request sent by Tyk Gateway. +- **secret_key** is the value of the secret used for signing. + +The function calls *generate_hmac_signature* to verify the signatures match. It returns true if the computed and client HMAC signatures match, otherwise false is returned. + +##### Set Error Response + +Add the following helper function to *tyk_async_server.py* to allow us to set the response status and error message if authentication fails. + +```python +def set_response_error(object: coprocess_object_pb2.Object, code: int, message: str) -> None: + object.request.return_overrides.response_code = code + object.request.return_overrides.response_error = message +``` + +Our function accepts the following three parameters: + +- **object** is an instance of the [Object]({{< ref "api-management/plugins/rich-plugins#object" >}}) message representing the payload sent by Tyk Gateway to the *Dispatcher* service in our gRPC server. For further details of the payload structure dispatched by Tyk Gateway to a gRPC server please consult our gRPC documentation. +- **code** is the HTTP status code to return in the response. +- **message** is the response message. + +The function modifies the *return_overrides* attribute of the request, updating the response status code and error message. The *return_overrides* attribute is an instance of a [ReturnOverrides]({{< ref "api-management/plugins/rich-plugins#returnoverrides" >}}) message that can be used to override the response of a given HTTP request. When this attribute is modified the request is terminated and is not sent upstream. + +##### Authenticate + +Add the following to your *tyk_async_server.py* file to implement the main custom authentication function. This parses the headers to extract the signature and date from the request, in addition to verifying the HMAC signature and key: + +```python +def authenticate(object: coprocess_object_pb2.Object) -> coprocess_object_pb2.Object: + keys_to_check = ["keyId", "algorithm", "signature"] + + auth_header = object.request.headers.get("Authorization") + date_header = object.request.headers.get("Date") + + parse_dict = parse_auth_header(auth_header) + + if not all(key in parse_dict for key in keys_to_check) or not all([auth_header, date_header]): + set_response_error(object, 400, "Custom middleware: Bad request") + return object + + try: + signature_valid = verify_hmac_signature( + parse_dict["algorithm"], + parse_dict["signature"], + date_header + ) + except ValueError: + set_response_error(object, 400, "Bad HMAC request, unsupported algorithm") + return object + + if not signature_valid or parse_dict["keyId"] != VALID_TOKEN: + set_response_error(object, 401, "Custom middleware: Not authorized") + else: + new_session = SessionState() + new_session.hmac_enabled = True + new_session.hmac_secret = SECRET + + object.metadata["token"] = VALID_TOKEN + object.session.CopyFrom(new_session) + + return object +``` + +The *Object* payload received from the Gateway is updated and returned as a response from the *Dispatcher* service: + +- If authentication fails then we set the error message and status code for the response accordingly, using our *set_response_error* function. +- If authentication passes then we update the session attribute in the *Object* payload to indicate that HMAC verification was performed and provide the secret used for signing. We also add the verified key to the meta data of the request payload. + +Specifically, our function performs the following tasks: + +- Extracts the *Date* and *Authorization* headers from the request and verifies that the *Authorization* header is structured correctly, using our *parse_auth_header* function. We store the extracted *Authorization* header fields in the *parse_dict* dictionary. If the structure is invalid then a 400 bad request response is returned to Tyk Gateway, using our *set_response_error* function. +- We use our *verify_hmac_signature* function to compute and verify the HMAC signature. A 400 bad request error is returned to the Gateway if HMAC signature verification fails, due to an unrecognized HMAC algorithm. +- A 401 unauthorized error response is returned to the Gateway under the following conditions: + + - The client HMAC signature and the computed HMAC signature do not match. + - The extracted key ID does not match the expected key value in VALID_TOKEN. + +- If HMAC signature verification passed and the key included in the *Authorization* header is valid then we update the *SessionState* instance to indicate that HMAC signature verification is enabled, i.e. *hmac_enabled* is set to true. We also specify the HMAC secret used for signing in the *hmac_secret* field and include the valid token in the metadata dictionary. + +##### Integrate Plugin + +Update the *Dispatch* method of the *PythonDispatcher* class in your *tyk_async_server.py* file so that our authenticate function is called when the a request is made by Tyk Gateway to execute a custom authentication (*HookType.CustomKeyCheck*) plugin. + +```python +class PythonDispatcher(coprocess_object_pb2_grpc.DispatcherServicer): + async def Dispatch( + self, object: coprocess_object_pb2.Object, context: grpc.aio.ServicerContext + ) -> coprocess_object_pb2.Object: + + logging.info(f"STATE for {object.hook_name}\n{MessageToJson(object)}\n") + + if object.hook_type == HookType.Pre: + logging.info(f"Pre plugin name: {object.hook_name}") + logging.info(f"Activated Pre Request plugin from API: {object.spec.get('APIID')}") + + elif object.hook_type == HookType.CustomKeyCheck: + logging.info(f"CustomAuth plugin: {object.hook_name}") + logging.info(f"Activated CustomAuth plugin from API: {object.spec.get('APIID')}") + + authenticate(object) + + elif object.hook_type == HookType.PostKeyAuth: + logging.info(f"PostKeyAuth plugin name: {object.hook_name}") + logging.info(f"Activated PostKeyAuth plugin from API: {object.spec.get('APIID')}") + + elif object.hook_type == HookType.Post: + logging.info(f"Post plugin name: {object.hook_name}") + logging.info(f"Activated Post plugin from API: {object.spec.get('APIID')}") + + elif object.hook_type == HookType.Response: + logging.info(f"Response plugin name: {object.hook_name}") + logging.info(f"Activated Response plugin from API: {object.spec.get('APIID')}") + logging.info("--------\n") + + return object +``` + +#### Test Plugin + +Create the following bash script, *hmac.sh*, to issue a test request to an API served by Tyk Gateway. The script computes a HMAC signature and constructs the *Authorization* and *Date* headers for a specified API. The *Authorization* header contains the HMAC signature and key for authentication. + +Replace the following constant values with values suitable for your environment: + +- **KEY** represents the key ID for the HMAC signed key that you created at the beginning of this guide. +- **HMAC_SECRET** represents the base64 encoded value of the secret for your HMAC key that you created at the beginning of this guide. +- **BASE_URL** represents the base URL, containing the protocol scheme, host and port number that Tyk Gateway listens to for API requests. +- **ENDPOINT** represents the path of your API that uses HMAC signed authentication. + +```bash +#!/bin/bash + +BASE_URL=http://localhost:8080 +ENDPOINT=/grpc-custom-auth/get +HMAC_ALGORITHM=hmac-sha512 +HMAC_SECRET=c2VjcmV0 +KEY=eyJvcmciOiI1ZTlkOTU0NGExZGNkNjAwMDFkMGVkMjAiLCJpZCI6ImdycGNfaG1hY19rZXkiLCJoIjoibXVybXVyNjQifQ== +REQUEST_URL=${BASE_URL}${ENDPOINT} + + +function urlencode() { + echo -n "$1" | perl -MURI::Escape -ne 'print uri_escape($_)' | sed "s/%20/+/g" +} + +# Set date in expected format +date="$(LC_ALL=C date -u +"%a, %d %b %Y %H:%M:%S GMT")" + +# Generate the signature using hmac algorithm with hmac secret from created Tyk key and +# then base64 encoded +signature=$(echo -n "date: ${date}" | openssl sha512 -binary -hmac "${HMAC_SECRET}" | base64) + +# Ensure the signature is base64 encoded +url_encoded_signature=$(echo -n "${signature}" | perl -MURI::Escape -ne 'print uri_escape($_)' | sed "s/%20/+/g") + +# Output the date, encoded date, signature and the url encoded signature +echo "request: ${REQUEST_URL}" +echo "date: $date" +echo "signature: $signature" +echo "url_encoded_signature: $url_encoded_signature" + +# Make the curl request using headers +printf "\n\n----\n\nMaking request to http://localhost:8080/grpc-custom-auth/get\n\n" +set -x +curl -v -H "Date: ${date}" \ + -H "Authorization: Signature keyId=\"${KEY}\",algorithm=\"${HMAC_ALGORITHM}\",signature=\"${url_encoded_signature}\"" \ + ${REQUEST_URL} +``` + +After creating and saving the script, ensure that it is executable by issuing the following command: + +```bash +chmod +x hmac.sh +``` + +Issue a test request by running the script: + +```bash +./hmac.sh +``` + +Observe the output of your gRPC server. You should see the request payload appear in the console output for the server and your custom authentication plugin should have been triggered. An illustrative example is given below: + +```bash +2024-05-13 12:53:49 INFO:root:STATE for CustomHMACCheck +2024-05-13 12:53:49 { +2024-05-13 12:53:49 "hookType": "CustomKeyCheck", +2024-05-13 12:53:49 "hookName": "CustomHMACCheck", +2024-05-13 12:53:49 "request": { +2024-05-13 12:53:49 "headers": { +2024-05-13 12:53:49 "User-Agent": "curl/8.1.2", +2024-05-13 12:53:49 "Date": "Mon, 13 May 2024 11:53:49 GMT", +2024-05-13 12:53:49 "Host": "localhost:8080", +2024-05-13 12:53:49 "Authorization": "Signature keyId=\"eyJvcmciOiI1ZTlkOTU0NGExZGNkNjAwMDFkMGVkMjAiLCJpZCI6ImdycGNfaG1hY19rZXkiLCJoIjoibXVybXVyNjQifQ==\",algorithm=\"hmac-sha512\",signature=\"e9OiifnTDgi3PW2EGJWfeQXCuhuhi6bGLiGhUTFpjEfgdKmX%2FQOFrePAQ%2FAoSFGU%2FzpP%2FCabmQi4zQDPdRh%2FZg%3D%3D\"", +2024-05-13 12:53:49 "Accept": "*/*" +2024-05-13 12:53:49 }, +2024-05-13 12:53:49 "url": "/grpc-custom-auth/get", +2024-05-13 12:53:49 "returnOverrides": { +2024-05-13 12:53:49 "responseCode": -1 +2024-05-13 12:53:49 }, +2024-05-13 12:53:49 "method": "GET", +2024-05-13 12:53:49 "requestUri": "/grpc-custom-auth/get", +2024-05-13 12:53:49 "scheme": "http" +2024-05-13 12:53:49 }, +2024-05-13 12:53:49 "spec": { +2024-05-13 12:53:49 "bundle_hash": "d41d8cd98f00b204e9800998ecf8427e", +2024-05-13 12:53:49 "OrgID": "5e9d9544a1dcd60001d0ed20", +2024-05-13 12:53:49 "APIID": "6c56dd4d3ad942a94474df6097df67ed" +2024-05-13 12:53:49 } +2024-05-13 12:53:49 } +2024-05-13 12:53:49 +2024-05-13 12:53:49 INFO:root:CustomAuth plugin: CustomHMACCheck +2024-05-13 12:53:49 INFO:root:Activated CustomAuth plugin from API: 6c56dd4d3ad942a94474df6097df67ed +2024-05-13 12:53:49 INFO:root:generating signature from: date: Mon, 13 May 2024 11:53:49 GMT +2024-05-13 12:53:49 INFO:root:Signatures matched! +2024-05-13 12:53:49 INFO:root:-------- +``` + +Try changing the SECRET and/or KEY constants with invalid values and observe the output of your gRPC server. You should notice that authentication fails. An illustrative example is given below: + +``` +2024-05-13 12:56:37 INFO:root:STATE for CustomHMACCheck +2024-05-13 12:56:37 { +2024-05-13 12:56:37 "hookType": "CustomKeyCheck", +2024-05-13 12:56:37 "hookName": "CustomHMACCheck", +2024-05-13 12:56:37 "request": { +2024-05-13 12:56:37 "headers": { +2024-05-13 12:56:37 "User-Agent": "curl/8.1.2", +2024-05-13 12:56:37 "Date": "Mon, 13 May 2024 11:56:37 GMT", +2024-05-13 12:56:37 "Host": "localhost:8080", +2024-05-13 12:56:37 "Authorization": "Signature keyId=\"eyJvcmciOiI1ZTlkOTU0NGExZGNkNjAwMDFkMGVkMjAiLCJpZCI6ImdycGNfaG1hY19rZXkiLCJoIjoibXVybXVyNjQifQ==\",algorithm=\"hmac-sha512\",signature=\"KXhkWOS01nbxuFfK7wEBggkydXlKJswxbukiplboJ2n%2BU6JiYOil%2Bx4OE4edWipg4EcG9T49nvY%2Fc9G0XFJcfg%3D%3D\"", +2024-05-13 12:56:37 "Accept": "*/*" +2024-05-13 12:56:37 }, +2024-05-13 12:56:37 "url": "/grpc-custom-auth/get", +2024-05-13 12:56:37 "returnOverrides": { +2024-05-13 12:56:37 "responseCode": -1 +2024-05-13 12:56:37 }, +2024-05-13 12:56:37 "method": "GET", +2024-05-13 12:56:37 "requestUri": "/grpc-custom-auth/get", +2024-05-13 12:56:37 "scheme": "http" +2024-05-13 12:56:37 }, +2024-05-13 12:56:37 "spec": { +2024-05-13 12:56:37 "bundle_hash": "d41d8cd98f00b204e9800998ecf8427e", +2024-05-13 12:56:37 "OrgID": "5e9d9544a1dcd60001d0ed20", +2024-05-13 12:56:37 "APIID": "6c56dd4d3ad942a94474df6097df67ed" +2024-05-13 12:56:37 } +2024-05-13 12:56:37 } +2024-05-13 12:56:37 +2024-05-13 12:56:37 INFO:root:CustomAuth plugin: CustomHMACCheck +2024-05-13 12:56:37 INFO:root:Activated CustomAuth plugin from API: 6c56dd4d3ad942a94474df6097df67ed +2024-05-13 12:56:37 INFO:root:generating signature from: date: Mon, 13 May 2024 11:56:37 GMT +2024-05-13 12:56:37 ERROR:root:Signatures did not match +2024-05-13 12:56:37 received: KXhkWOS01nbxuFfK7wEBggkydXlKJswxbukiplboJ2n+U6JiYOil+x4OE4edWipg4EcG9T49nvY/c9G0XFJcfg== +2024-05-13 12:56:37 expected: zT17C2tgDCYBJCgFFN/mknf6XydPaV98a5gMPNUHYxZyYwYedIPIhyDRQsMF9GTVFe8khCB1FhfyhpmzrUR2Lw== +``` + +#### Summary + +In this guide, we've explained how to write a Python gRPC custom authentication plugin for Tyk Gateway, using HMAC-signed authentication as a practical example. Through clear instructions and code examples, we've provided developers with insights into the process of creating custom authentication logic tailored to their specific API authentication needs. + +While Tyk Gateway already supports HMAC-signed authentication out of the box, this guide goes beyond basic implementation by demonstrating how to extend its capabilities through custom plugins. By focusing on HMAC-signed authentication, developers have gained valuable experience in crafting custom authentication mechanisms that can be adapted and expanded to meet diverse authentication requirements. + +It's important to note that the authentication mechanism implemented in this guide solely verifies the HMAC signature's validity and does not include access control checks against specific API resources. Developers should enhance this implementation by integrating access control logic to ensure authenticated requests have appropriate access permissions. + +By mastering the techniques outlined in this guide, developers are better equipped to address complex authentication challenges and build robust API security architectures using Tyk Gateway's extensibility features. This guide serves as a foundation for further exploration and experimentation with custom authentication plugins, empowering developers to innovate and customize API authentication solutions according to their unique requirements. + +--- + +
+ +### Performance + +These are some benchmarks performed on gRPC plugins. + +gRPC plugins may use different transports, we've tested TCP and Unix Sockets. + +#### TCP + +{{< img src="/img/diagrams/tcpResponseTime.png" alt="TCP Response Times" >}} + +{{< img src="/img/diagrams/tcpHitRate.png" alt="TCP Hit Rate" >}} + +#### Unix Socket + +{{< img src="/img/diagrams/unixResponseTime.png" alt="Unix Socket Response Times" >}} + +{{< img src="/img/diagrams/unixHitRate.png" alt="Unix Socket Hit Rate" >}} + +## Using Lua + +### Overview + +#### Requirements + +Tyk uses [LuaJIT](http://luajit.org/). The main requirement is the LuaJIT shared library, you may find this as `libluajit-x` in most distros. + +For Ubuntu 14.04 you may use: + +`$ apt-get install libluajit-5.1-2 +$ apt-get install luarocks` + +The LuaJIT required modules are as follows: + +* [lua-cjson](https://github.com/mpx/lua-cjson): in case you have `luarocks`, run: `$ luarocks install lua-cjson` + +#### How to write LuaJIT Plugins + +We have a demo plugin hosted in the repo [tyk-plugin-demo-lua](https://github.com/TykTechnologies/tyk-plugin-demo-lua). The project implements a simple middleware for header injection, using a Pre hook (see [Tyk custom middleware hooks]({{< ref "api-management/plugins/javascript#using-javascript-with-tyk" >}})) and [mymiddleware.lua](https://github.com/TykTechnologies/tyk-plugin-demo-lua/blob/master/mymiddleware.lua). +#### Lua Performance +Lua support is currently in beta stage. We are planning performance optimizations for future releases. +#### Tyk Lua API Methods +Tyk Lua API methods aren’t currently supported. + +### Lua Plugin Tutorial + +#### Settings in the API Definition + +To add a Lua plugin to your API, you must specify the bundle name using the `custom_middleware_bundle` field: + +```json +{ + "name": "Tyk Test API", + "api_id": "1", + "org_id": "default", + "definition": { + "location": "header", + "key": "version" + }, + "auth": { + "auth_header_name": "authorization" + }, + "use_keyless": true, + "version_data": { + "not_versioned": true, + "versions": { + "Default": { + "name": "Default", + "expires": "3000-01-02 15:04", + "use_extended_paths": true, + "extended_paths": { + "ignored": [], + "white_list": [], + "black_list": [] + } + } + } + }, + "proxy": { + "listen_path": "/quickstart/", + "target_url": "http://httpbin.org", + "strip_listen_path": true + }, + "custom_middleware_bundle": "test-bundle", +} +``` + +#### Global settings + +To enable Lua plugins you need to add the following block to `tyk.conf`: + +```json +"coprocess_options": { + "enable_coprocess": true, +}, +"enable_bundle_downloader": true, +"bundle_base_url": "http://my-bundle-server.com/bundles/", +"public_key_path": "/path/to/my/pubkey", +``` + +`enable_coprocess` enables the rich plugins feature. + +`enable_bundle_downloader` enables the bundle downloader. + +`bundle_base_url` is a base URL that will be used to download the bundle, in this example we have "test-bundle" specified in the API settings, Tyk will fetch the following URL: `http://my-bundle-server.com/bundles/test-bundle`. + +`public_key_path` sets a public key, this is used for verifying signed bundles, you may omit this if unsigned bundles are used. + +#### Running the Tyk Lua build + +To use Tyk with Lua support you will need to use an alternative binary, it is provided in the standard Tyk package but it has a different service name. + +Firstly stop the standard Tyk version: + +```console +service tyk-gateway stop +``` + +and then start the Lua build: + +```console +service tyk-gateway-lua start +``` + + diff --git a/tyk-docs/content/api-management/rate-limit.md b/tyk-docs/content/api-management/rate-limit.md index 310f87f95e..d3b190ad35 100644 --- a/tyk-docs/content/api-management/rate-limit.md +++ b/tyk-docs/content/api-management/rate-limit.md @@ -438,7 +438,7 @@ Disabling the rate limiter at the API-Level does not disable rate limiting at th #### Can I set rate limits by IP address? -Not yet, though IP-based rate limiting is possible using custom pre-processor middleware JavaScript that generates tokens based on IP addresses. See our [Middleware Scripting Guide]({{< ref "plugins/supported-languages/javascript-middleware/middleware-scripting-guide" >}}) for more details. +Not yet, though IP-based rate limiting is possible using custom pre-processor middleware JavaScript that generates tokens based on IP addresses. See our [Middleware Scripting Guide]({{< ref "api-management/plugins/javascript#using-javascript-with-tyk" >}}) for more details. ## Rate Limiting by API diff --git a/tyk-docs/content/api-management/security-best-practices.md b/tyk-docs/content/api-management/security-best-practices.md index 91076ae8bb..1467aab8eb 100644 --- a/tyk-docs/content/api-management/security-best-practices.md +++ b/tyk-docs/content/api-management/security-best-practices.md @@ -118,7 +118,7 @@ Tyk offers several mechanisms to help protect an API from Security Misconfigurat - [Schema Introspection]({{< ref "graphql/introspection" >}}) ensures that the Tyk Dashboard automatically uses the schema of the upstream GraphQL API and can keep it synchronised if it changes. - [GraphQL Schema Validation]({{< ref "graphql/validation#schema-validation" >}}) prevents invalid schemas from being saved. This catches errors such as duplicate type names and usage of unknown types. - Third-party [Secret Storage]({{< ref "tyk-self-managed#manage-multi-environment-and-distributed-setups" >}}) to centralise configuration of sensitive data such as passwords. This data can then be dynamically referenced by Tyk configuration files, rather than being hard coded. -- Users can can write their own [custom plugins]({{< ref "plugins" >}}) in a variety of languages, either directly or through gRPC calls, to implement their requirements. +- Users can can write their own [custom plugins]({{< ref "api-management/plugins/overview#" >}}) in a variety of languages, either directly or through gRPC calls, to implement their requirements. The Ops team should also take reponsibility for monitoring the APIs for errors and patching accordingly. Regular [Penetration Tests](https://en.wikipedia.org/wiki/Penetration_test) should be scheduled to ensure the security of published services. Tyk, through our Professional Services or Partners, can assist in the process. diff --git a/tyk-docs/content/basic-config-and-security/report-monitor-trigger-events/custom-handlers-javascript.md b/tyk-docs/content/basic-config-and-security/report-monitor-trigger-events/custom-handlers-javascript.md index c3d4ad9c65..003b177c7a 100755 --- a/tyk-docs/content/basic-config-and-security/report-monitor-trigger-events/custom-handlers-javascript.md +++ b/tyk-docs/content/basic-config-and-security/report-monitor-trigger-events/custom-handlers-javascript.md @@ -7,9 +7,9 @@ description: "How to create your own custom event handlers in JavaScript" Tyk supports you to script your own custom code in JavaScript (JS) that will be invoked in response to API events. This is executed asynchronously so you don't need to worry about it blocking the Gateway handling requests. Event handlers like this can be very powerful for automating session, user and API-level functions. -It is important to note that unlike custom JavaScript [plugins]({{< ref "plugins/supported-languages/javascript-middleware" >}}), custom event handlers execute in a *global* JavaScript environment. This means that you need to be careful when naming the event handlers: if you use the same event handler name for different event handling code across two APIs, only one of them will execute, as the other will be overridden when loaded. +It is important to note that unlike custom JavaScript [plugins]({{< ref "api-management/plugins/javascript#" >}}), custom event handlers execute in a *global* JavaScript environment. This means that you need to be careful when naming the event handlers: if you use the same event handler name for different event handling code across two APIs, only one of them will execute, as the other will be overridden when loaded. -Custom event handlers have access to the [JavaScript API]({{< ref "plugins/supported-languages/javascript-middleware/javascript-api" >}}) which gives access to the session object and enables your code to make HTTP calls. This is particularly useful if you want to interface with another API with a complex request/response cycle. +Custom event handlers have access to the [JavaScript API]({{< ref "api-management/plugins/javascript#javascript-api" >}}) which gives access to the session object and enables your code to make HTTP calls. This is particularly useful if you want to interface with another API with a complex request/response cycle.
{{< note success >}} @@ -22,7 +22,7 @@ Custom event handlers are currently only supported by Tyk Classic APIs. A custom event handler consists of a function that accepts two variables (`event` and `context`) and has no return value. -Creating an event handler is very similar to [creating custom JS plugins]({{< ref "plugins/supported-languages/javascript-middleware/middleware-scripting-guide" >}}), simply invoke the correct constructors with a closure in the TykJS namespace: +Creating an event handler is very similar to [creating custom JS plugins]({{< ref "api-management/plugins/javascript#using-javascript-with-tyk" >}}), simply invoke the correct constructors with a closure in the TykJS namespace: ```js // ---- Sample custom event handler ----- diff --git a/tyk-docs/content/developer-support/release-notes/archived.md b/tyk-docs/content/developer-support/release-notes/archived.md index 5ca92acb39..a5cd1e4891 100644 --- a/tyk-docs/content/developer-support/release-notes/archived.md +++ b/tyk-docs/content/developer-support/release-notes/archived.md @@ -87,7 +87,7 @@ func AddFooBarHeader(rw http.ResponseWriter, r *http.Request) { } ``` -See our [Golang plugin documentation]({{< ref "plugins/supported-languages/golang" >}}) for more details. +See our [Golang plugin documentation]({{< ref "api-management/plugins/golang#" >}}) for more details. ### Distributed tracing @@ -784,8 +784,8 @@ We have added support for specifying allowed SSL ciphers using the following op * The JSVM `spec` object now has access to `APIID` and `OriginID` to reflect similar functionality of Coprocess plugins. * Plugins now have access to Host HTTP Header. -[JSVM Docs]({{< ref "plugins/supported-languages/javascript-middleware/middleware-scripting-guide" >}}) -[Plugin Data Structure Docs]({{< ref "plugins/supported-languages/rich-plugins/rich-plugins-data-structures" >}}) +[JSVM Docs]({{< ref "api-management/plugins/javascript#using-javascript-with-tyk" >}}) +[Plugin Data Structure Docs]({{< ref "api-management/plugins/rich-plugins#rich-plugins-data-structures" >}}) ### Tyk Dashboard v1.5.0 diff --git a/tyk-docs/content/developer-support/release-notes/dashboard.md b/tyk-docs/content/developer-support/release-notes/dashboard.md index be0ce6138a..6612ff674e 100644 --- a/tyk-docs/content/developer-support/release-notes/dashboard.md +++ b/tyk-docs/content/developer-support/release-notes/dashboard.md @@ -67,7 +67,7 @@ There are no breaking changes in this release. | Third Party Dependency | Tested Versions | Compatible Versions | Comments | | ---------------------------------------------------------- | ---------------------- | ---------------------- | -------- | -| [GoLang](https://go.dev/dl/) | 1.22 | 1.22 | [Go plugins]({{< ref "plugins/supported-languages/golang" >}}) must be built using Go 1.22 | +| [GoLang](https://go.dev/dl/) | 1.22 | 1.22 | [Go plugins]({{< ref "api-management/plugins/golang#" >}}) must be built using Go 1.22 | | [Redis](https://redis.io/download/) | 6.2.x, 7.x | 6.2.x, 7.x | Used by Tyk Dashboard | | [MongoDB](https://www.mongodb.com/try/download/community) | 5.0.x, 6.0.x, 7.0.x | 5.0.x, 6.0.x, 7.0.x | Used by Tyk Dashboard | | [PostgreSQL](https://www.postgresql.org/download/) | 12.x - 16.x LTS | 12.x - 16.x | Used by Tyk Dashboard | @@ -157,7 +157,7 @@ An example is given below for illustrative purposes only. Tested Versions and Co | Third Party Dependency | Tested Versions | Compatible Versions | Comments | | ---------------------------------------------------------- | ---------------------- | ---------------------- | -------- | -| [GoLang](https://go.dev/dl/) | 1.22 | 1.22 | [Go plugins]({{< ref "plugins/supported-languages/golang" >}}) must be built using Go 1.22 | +| [GoLang](https://go.dev/dl/) | 1.22 | 1.22 | [Go plugins]({{< ref "api-management/plugins/golang#" >}}) must be built using Go 1.22 | | [Redis](https://redis.io/download/) | 6.2.x, 7.x | 6.2.x, 7.x | Used by Tyk Dashboard | | [MongoDB](https://www.mongodb.com/try/download/community) | 5.0.x, 6.0.x, 7.0.x | 5.0.x, 6.0.x, 7.0.x | Used by Tyk Dashboard | | [PostgreSQL](https://www.postgresql.org/download/) | 12.x - 16.x LTS | 12.x - 16.x | Used by Tyk Dashboard | @@ -400,7 +400,7 @@ An example is given below for illustrative purposes only. Tested Versions and Co | Third Party Dependency | Tested Versions | Compatible Versions | Comments | | ---------------------------------------------------------- | ---------------------- | ---------------------- | -------- | -| [GoLang](https://go.dev/dl/) | 1.22 | 1.22 | [Go plugins]({{< ref "plugins/supported-languages/golang" >}}) must be built using Go 1.22 | +| [GoLang](https://go.dev/dl/) | 1.22 | 1.22 | [Go plugins]({{< ref "api-management/plugins/golang#" >}}) must be built using Go 1.22 | | [Redis](https://redis.io/download/) | 6.2.x, 7.x | 6.2.x, 7.x | Used by Tyk Dashboard | | [MongoDB](https://www.mongodb.com/try/download/community) | 5.0.x, 6.0.x, 7.0.x | 5.0.x, 6.0.x, 7.0.x | Used by Tyk Dashboard | | [PostgreSQL](https://www.postgresql.org/download/) | 12.x - 16.x LTS | 12.x - 16.x | Used by Tyk Dashboard | @@ -479,7 +479,7 @@ An example is given below for illustrative purposes only. Tested Versions and Co | Third Party Dependency | Tested Versions | Compatible Versions | Comments | | ---------------------------------------------------------- | ---------------------- | ---------------------- | -------- | -| [GoLang](https://go.dev/dl/) | 1.22 | 1.22 | [Go plugins]({{< ref "plugins/supported-languages/golang" >}}) must be built using Go 1.22 | +| [GoLang](https://go.dev/dl/) | 1.22 | 1.22 | [Go plugins]({{< ref "api-management/plugins/golang#" >}}) must be built using Go 1.22 | | [Redis](https://redis.io/download/) | 6.2.x, 7.x | 6.2.x, 7.x | Used by Tyk Dashboard | | [MongoDB](https://www.mongodb.com/try/download/community) | 5.0.x, 6.0.x, 7.0.x | 5.0.x, 6.0.x, 7.0.x | Used by Tyk Dashboard | | [PostgreSQL](https://www.postgresql.org/download/) | 12.x - 16.x LTS | 12.x - 16.x | Used by Tyk Dashboard | @@ -672,7 +672,7 @@ There are no breaking changes in this release. | Third Party Dependency | Tested Versions | Compatible Versions | Comments | | ---------------------------------------------------------- | ---------------------- | ---------------------- | -------- | -| [GoLang](https://go.dev/dl/) | 1.21 | 1.21 | [Go plugins]({{< ref "plugins/supported-languages/golang" >}}) must be built using Go 1.21 | +| [GoLang](https://go.dev/dl/) | 1.21 | 1.21 | [Go plugins]({{< ref "api-management/plugins/golang#" >}}) must be built using Go 1.21 | | [Redis](https://redis.io/download/) | 6.2.x, 7.x | 6.2.x, 7.x | Used by Tyk Dashboard | | [MongoDB](https://www.mongodb.com/try/download/community) | 5.0.x, 6.0.x, 7.0.x | 5.0.x, 6.0.x, 7.0.x | Used by Tyk Dashboard | | [PostgreSQL](https://www.postgresql.org/download/) | 12.x - 16.x LTS | 12.x - 16.x | Used by Tyk Dashboard | @@ -730,7 +730,7 @@ There are no breaking changes in this release. | Third Party Dependency | Tested Versions | Compatible Versions | Comments | | ---------------------------------------------------------- | ---------------------- | ---------------------- | -------- | -| [GoLang](https://go.dev/dl/) | 1.21 | 1.21 | [Go plugins]({{< ref "plugins/supported-languages/golang" >}}) must be built using Go 1.21 | +| [GoLang](https://go.dev/dl/) | 1.21 | 1.21 | [Go plugins]({{< ref "api-management/plugins/golang#" >}}) must be built using Go 1.21 | | [Redis](https://redis.io/download/) | 6.2.x, 7.x | 6.2.x, 7.x | Used by Tyk Dashboard | | [MongoDB](https://www.mongodb.com/try/download/community) | 5.0.x, 6.0.x, 7.0.x | 5.0.x, 6.0.x, 7.0.x | Used by Tyk Dashboard | | [PostgreSQL](https://www.postgresql.org/download/) | 12.x - 16.x LTS | 12.x - 16.x | Used by Tyk Dashboard | @@ -805,7 +805,7 @@ An example is given below for illustrative purposes only. Tested Versions and Co | Third Party Dependency | Tested Versions | Compatible Versions | Comments | | ---------------------------------------------------------- | ---------------------- | ---------------------- | -------- | -| [GoLang](https://go.dev/dl/) | 1.21 | 1.21 | [Go plugins]({{< ref "plugins/supported-languages/golang" >}}) must be built using Go 1.21 | +| [GoLang](https://go.dev/dl/) | 1.21 | 1.21 | [Go plugins]({{< ref "api-management/plugins/golang#" >}}) must be built using Go 1.21 | | [Redis](https://redis.io/download/) | 6.2.x, 7.x | 6.2.x, 7.x | Used by Tyk Dashboard | | [MongoDB](https://www.mongodb.com/try/download/community) | 5.0.x, 6.0.x, 7.0.x | 5.0.x, 6.0.x, 7.0.x | Used by Tyk Dashboard | | [PostgreSQL](https://www.postgresql.org/download/) | 12.x - 16.x LTS | 12.x - 16.x | Used by Tyk Dashboard | @@ -972,7 +972,7 @@ An example is given below for illustrative purposes only. Tested Versions and Co | Third Party Dependency | Tested Versions | Compatible Versions | Comments | | ---------------------------------------------------------- | ---------------------- | ---------------------- | -------- | -| [GoLang](https://go.dev/dl/) | 1.21 | 1.21 | [Go plugins]({{< ref "plugins/supported-languages/golang" >}}) must be built using Go 1.21 | +| [GoLang](https://go.dev/dl/) | 1.21 | 1.21 | [Go plugins]({{< ref "api-management/plugins/golang#" >}}) must be built using Go 1.21 | | [Redis](https://redis.io/download/) | 6.2.x, 7.x | 6.2.x, 7.x | Used by Tyk Dashboard | | [MongoDB](https://www.mongodb.com/try/download/community) | 5.0.x, 6.0.x, 7.0.x | 5.0.x, 6.0.x, 7.0.x | Used by Tyk Dashboard | | [PostgreSQL](https://www.postgresql.org/download/) | 12.x - 16.x LTS | 12.x - 16.x | Used by Tyk Dashboard | @@ -1180,7 +1180,7 @@ An example is given below for illustrative purposes only. Tested Versions and Co | Third Party Dependency | Tested Versions | Compatible Versions | Comments | | ---------------------------------------------------------- | ---------------------- | ---------------------- | -------- | -| [GoLang](https://go.dev/dl/) | 1.22 | 1.22 | [Go plugins]({{< ref "plugins/supported-languages/golang" >}}) must be built using Go 1.22 | +| [GoLang](https://go.dev/dl/) | 1.22 | 1.22 | [Go plugins]({{< ref "api-management/plugins/golang#" >}}) must be built using Go 1.22 | | [Redis](https://redis.io/download/) | 6.2.x, 7.x | 6.2.x, 7.x | Used by Tyk Dashboard | | [MongoDB](https://www.mongodb.com/try/download/community) | 5.0.x, 6.0.x, 7.0.x | 5.0.x, 6.0.x, 7.0.x | Used by Tyk Dashboard | | [PostgreSQL](https://www.postgresql.org/download/) | 12.x - 16.x LTS | 12.x - 16.x | Used by Tyk Dashboard | @@ -1278,7 +1278,7 @@ An example is given below for illustrative purposes only. Tested Versions and Co | Third Party Dependency | Tested Versions | Compatible Versions | Comments | | ---------------------------------------------------------- | ---------------------- | ---------------------- | -------- | -| [GoLang](https://go.dev/dl/) | 1.22 | 1.22 | [Go plugins]({{< ref "plugins/supported-languages/golang" >}}) must be built using Go 1.22 | +| [GoLang](https://go.dev/dl/) | 1.22 | 1.22 | [Go plugins]({{< ref "api-management/plugins/golang#" >}}) must be built using Go 1.22 | | [Redis](https://redis.io/download/) | 6.2.x, 7.x | 6.2.x, 7.x | Used by Tyk Dashboard | | [MongoDB](https://www.mongodb.com/try/download/community) | 5.0.x, 6.0.x, 7.0.x | 5.0.x, 6.0.x, 7.0.x | Used by Tyk Dashboard | | [PostgreSQL](https://www.postgresql.org/download/) | 12.x - 16.x LTS | 12.x - 16.x | Used by Tyk Dashboard | @@ -1381,7 +1381,7 @@ An example is given below for illustrative purposes only. Tested Versions and Co | Third Party Dependency | Tested Versions | Compatible Versions | Comments | | ---------------------------------------------------------- | ---------------------- | ---------------------- | -------- | -| [GoLang](https://go.dev/dl/) | 1.22 | 1.22 | [Go plugins]({{< ref "plugins/supported-languages/golang" >}}) must be built using Go 1.22 | +| [GoLang](https://go.dev/dl/) | 1.22 | 1.22 | [Go plugins]({{< ref "api-management/plugins/golang#" >}}) must be built using Go 1.22 | | [Redis](https://redis.io/download/) | 6.2.x, 7.x | 6.2.x, 7.x | Used by Tyk Dashboard | | [MongoDB](https://www.mongodb.com/try/download/community) | 5.0.x, 6.0.x, 7.0.x | 5.0.x, 6.0.x, 7.0.x | Used by Tyk Dashboard | | [PostgreSQL](https://www.postgresql.org/download/) | 12.x - 16.x LTS | 12.x - 16.x | Used by Tyk Dashboard | @@ -1461,7 +1461,7 @@ An example is given below for illustrative purposes only. Tested Versions and Co | Third Party Dependency | Tested Versions | Compatible Versions | Comments | | ---------------------------------------------------------- | ---------------------- | ---------------------- | -------- | -| [GoLang](https://go.dev/dl/) | 1.22 | 1.22 | [Go plugins]({{< ref "plugins/supported-languages/golang" >}}) must be built using Go 1.22 | +| [GoLang](https://go.dev/dl/) | 1.22 | 1.22 | [Go plugins]({{< ref "api-management/plugins/golang#" >}}) must be built using Go 1.22 | | [Redis](https://redis.io/download/) | 6.2.x, 7.x | 6.2.x, 7.x | Used by Tyk Dashboard | | [MongoDB](https://www.mongodb.com/try/download/community) | 5.0.x, 6.0.x, 7.0.x | 5.0.x, 6.0.x, 7.0.x | Used by Tyk Dashboard | | [PostgreSQL](https://www.postgresql.org/download/) | 12.x - 16.x LTS | 12.x - 16.x | Used by Tyk Dashboard | @@ -1635,7 +1635,7 @@ An example is given below for illustrative purposes only. Tested Versions and Co | Third Party Dependency | Tested Versions | Compatible Versions | Comments | | ---------------------------------------------------------- | ---------------------- | ---------------------- | -------- | -| [GoLang](https://go.dev/dl/) | 1.21 | 1.21 | [Go plugins]({{< ref "plugins/supported-languages/golang" >}}) must be built using Go 1.21 | +| [GoLang](https://go.dev/dl/) | 1.21 | 1.21 | [Go plugins]({{< ref "api-management/plugins/golang#" >}}) must be built using Go 1.21 | | [Redis](https://redis.io/download/) | 6.2.x, 7.x | 6.2.x, 7.x | Used by Tyk Dashboard | | [MongoDB](https://www.mongodb.com/try/download/community) | 5.0.x, 6.0.x, 7.0.x | 5.0.x, 6.0.x, 7.0.x | Used by Tyk Dashboard | | [PostgreSQL](https://www.postgresql.org/download/) | 12.x - 16.x LTS | 12.x - 16.x | Used by Tyk Dashboard | @@ -1706,7 +1706,7 @@ An example is given below for illustrative purposes only. Tested Versions and Co | Third Party Dependency | Tested Versions | Compatible Versions | Comments | | ---------------------------------------------------------- | ---------------------- | ---------------------- | -------- | -| [GoLang](https://go.dev/dl/) | 1.21 | 1.21 | [Go plugins]({{< ref "plugins/supported-languages/golang" >}}) must be built using Go 1.21 | +| [GoLang](https://go.dev/dl/) | 1.21 | 1.21 | [Go plugins]({{< ref "api-management/plugins/golang#" >}}) must be built using Go 1.21 | | [Redis](https://redis.io/download/) | 6.2.x, 7.x | 6.2.x, 7.x | Used by Tyk Dashboard | | [MongoDB](https://www.mongodb.com/try/download/community) | 5.0.x, 6.0.x, 7.0.x | 5.0.x, 6.0.x, 7.0.x | Used by Tyk Dashboard | | [PostgreSQL](https://www.postgresql.org/download/) | 12.x - 16.x LTS | 12.x - 16.x | Used by Tyk Dashboard | @@ -1820,7 +1820,7 @@ An example is given below for illustrative purposes only. Tested Versions and Co | Third Party Dependency | Tested Versions | Compatible Versions | Comments | | ---------------------------------------------------------- | ---------------------- | ---------------------- | -------- | -| [GoLang](https://go.dev/dl/) | 1.21 | 1.21 | [Go plugins]({{< ref "plugins/supported-languages/golang" >}}) must be built using Go 1.21 | +| [GoLang](https://go.dev/dl/) | 1.21 | 1.21 | [Go plugins]({{< ref "api-management/plugins/golang#" >}}) must be built using Go 1.21 | | [Redis](https://redis.io/download/) | 6.2.x, 7.x | 6.2.x, 7.x | Used by Tyk Dashboard | | [MongoDB](https://www.mongodb.com/try/download/community) | 5.0.x, 6.0.x, 7.0.x | 5.0.x, 6.0.x, 7.0.x | Used by Tyk Dashboard | | [PostgreSQL](https://www.postgresql.org/download/) | 12.x - 16.x LTS | 12.x - 16.x | Used by Tyk Dashboard | @@ -1960,7 +1960,7 @@ An example is given below for illustrative purposes only. Tested Versions and Co | Third Party Dependency | Tested Versions | Compatible Versions | Comments | | ---------------------------------------------------------- | ---------------------- | ---------------------- | -------- | -| [GoLang](https://go.dev/dl/) | 1.21 | 1.21 | [Go plugins]({{< ref "plugins/supported-languages/golang" >}}) must be built using Go 1.21 | +| [GoLang](https://go.dev/dl/) | 1.21 | 1.21 | [Go plugins]({{< ref "api-management/plugins/golang#" >}}) must be built using Go 1.21 | | [Redis](https://redis.io/download/) | 6.2.x, 7.x | 6.2.x, 7.x | Used by Tyk Dashboard | | [MongoDB](https://www.mongodb.com/try/download/community) | 5.0.x, 6.0.x, 7.0.x | 5.0.x, 6.0.x, 7.0.x | Used by Tyk Dashboard | | [PostgreSQL](https://www.postgresql.org/download/) | 12.x - 16.x LTS | 12.x - 16.x | Used by Tyk Dashboard | @@ -2094,7 +2094,7 @@ An example is given below for illustrative purposes only. Tested Versions and Co | Third Party Dependency | Tested Versions | Compatible Versions | Comments | | ---------------------------------------------------------- | ---------------------- | ---------------------- | -------- | -| [GoLang](https://go.dev/dl/) | 1.21 | 1.21 | [Go plugins]({{< ref "plugins/supported-languages/golang" >}}) must be built using Go 1.21 | +| [GoLang](https://go.dev/dl/) | 1.21 | 1.21 | [Go plugins]({{< ref "api-management/plugins/golang#" >}}) must be built using Go 1.21 | | [Redis](https://redis.io/download/) | 6.2.x, 7.x | 6.2.x, 7.x | Used by Tyk Dashboard | | [MongoDB](https://www.mongodb.com/try/download/community) | 5.0.x, 6.0.x, 7.0.x | 5.0.x, 6.0.x, 7.0.x | Used by Tyk Dashboard | | [PostgreSQL](https://www.postgresql.org/download/) | 11.x - 15.x LTS | 11.x - 15.x | Used by Tyk Dashboard | @@ -2287,7 +2287,7 @@ An example is given below for illustrative purposes only. Tested Versions and Co | Third Party Dependency | Tested Versions | Compatible Versions | Comments | | ---------------------------------------------------------- | ---------------------- | ---------------------- | -------- | -| [GoLang](https://go.dev/dl/) | 1.21 | 1.21 | [Go plugins]({{< ref "plugins/supported-languages/golang" >}}) must be built using Go 1.21 | +| [GoLang](https://go.dev/dl/) | 1.21 | 1.21 | [Go plugins]({{< ref "api-management/plugins/golang#" >}}) must be built using Go 1.21 | | [Redis](https://redis.io/download/) | 6.2.x, 7.x | 6.2.x, 7.x | Used by Tyk Dashboard | | [MongoDB](https://www.mongodb.com/try/download/community) | 5.0.x, 6.0.x, 7.0.x | 5.0.x, 6.0.x, 7.0.x | Used by Tyk Dashboard | | [PostgreSQL](https://www.postgresql.org/download/) | 11.x - 15.x LTS | 11.x - 15.x | Used by Tyk Dashboard | @@ -3206,7 +3206,7 @@ Fixed an issue with *MongoDB* connection strings. To ensure consistent compatibi **Attention warning*: Please read carefully this section. We have two topics to report: ###### Golang Version upgrade -Our Dashboard is using [Golang 1.19](https://tip.golang.org/doc/go1.19) programming language starting with the 5.1 release. This brings improvements to the code base and allows us to benefit from the latest features and security enhancements in Go. Don’t forget that, if you’re using GoPlugins, you'll need to [recompile]({{< ref "plugins/supported-languages/golang#upgrading-your-tyk-gateway" >}}) these to maintain compatibility with the latest Gateway. +Our Dashboard is using [Golang 1.19](https://tip.golang.org/doc/go1.19) programming language starting with the 5.1 release. This brings improvements to the code base and allows us to benefit from the latest features and security enhancements in Go. Don’t forget that, if you’re using GoPlugins, you'll need to [recompile]({{< ref "api-management/plugins/golang#upgrading-your-tyk-gateway" >}}) these to maintain compatibility with the latest Gateway. ###### Tyk OAS APIs To provide a superior experience with OAS APIs, we have made some changes which include various security fixes, improved validation etc. Upgrading to v5.1 from v4.x.x may be irreversible, rollback to v4.x.x could break your OAS API definitions. For this reason, we recommend making a database backup so you can always restore from the backup (of v4.X.X) in case you encounter a problem during the upgrade. Please refer to our guides for detailed information on [upgrading Tyk]({{}}) and [how to back up tyk]({{}}) @@ -3640,7 +3640,7 @@ If you want switch from MongoDB to SQL, you can [use our migration tool]({{< ref {{< note success >}} **Note** -Note: Upgrading the Golang version implies that all the Golang custom plugins that you are using need to be recompiled before migrating to 4.3 version of the Gateway. Check our docs for more details [Golang Plugins]({{< ref "/content/plugins/supported-languages/golang.md" >}}). +Note: Upgrading the Golang version implies that all the Golang custom plugins that you are using need to be recompiled before migrating to 4.3 version of the Gateway. Check our docs for more details [Golang Plugins]({{< ref "api-management/plugins/golang" >}}). {{< /note >}} ## 4.2 Release Notes @@ -4059,7 +4059,7 @@ Want to reference secrets from a KV store in your API definitions? We now have n We added a new middleware hook allowing middleware to modify the response from the upstream. Using response middleware you can transform, inspect or obfuscate parts of the response body or response headers, or fire an event or webhook based on information received by the upstream service. -At the moment the Response hook is supported for [Python and gRPC plugins]({{< ref "plugins/supported-languages/rich-plugins/rich-plugins-work#coprocess-dispatcher---hooks" >}}). +At the moment the Response hook is supported for [Python and gRPC plugins]({{< ref "api-management/plugins/rich-plugins#coprocess-dispatcher---hooks" >}}). ##### Enhanced Gateway health check API diff --git a/tyk-docs/content/developer-support/release-notes/gateway.md b/tyk-docs/content/developer-support/release-notes/gateway.md index 6d95af7f79..a842087787 100644 --- a/tyk-docs/content/developer-support/release-notes/gateway.md +++ b/tyk-docs/content/developer-support/release-notes/gateway.md @@ -78,7 +78,7 @@ An illustrative example is shown below. --> | Third Party Dependency | Tested Versions | Compatible Versions | Comments | | ------------------------------------------------------------ | ---------------------- | ---------------------- | -------- | -| [Go](https://go.dev/dl/) | 1.22 | 1.22 | [Go plugins]({{< ref "/plugins/supported-languages/golang" >}}) must be built using Go 1.22 | +| [Go](https://go.dev/dl/) | 1.22 | 1.22 | [Go plugins]({{< ref "api-management/plugins/golang#" >}}) must be built using Go 1.22 | | [Redis](https://redis.io/download/) | 6.2.x, 7.x | 6.2.x, 7.x | Used by Tyk Gateway | | [OpenAPI Specification](https://spec.openapis.org/oas/v3.0.3)| v3.0.x | v3.0.x | Supported by [Tyk OAS]({{< ref "api-management/gateway-config-tyk-oas#tyk-oas-api-definition-object" >}}) | @@ -236,7 +236,7 @@ An illustrative example is shown below. --> | Third Party Dependency | Tested Versions | Compatible Versions | Comments | | ------------------------------------------------------------ | ---------------------- | ---------------------- | -------- | -| [Go](https://go.dev/dl/) | 1.22 | 1.22 | [Go plugins]({{< ref "/plugins/supported-languages/golang" >}}) must be built using Go 1.22 | +| [Go](https://go.dev/dl/) | 1.22 | 1.22 | [Go plugins]({{< ref "api-management/plugins/golang#" >}}) must be built using Go 1.22 | | [Redis](https://redis.io/download/) | 6.2.x, 7.x | 6.2.x, 7.x | Used by Tyk Gateway | | [OpenAPI Specification](https://spec.openapis.org/oas/v3.0.3)| v3.0.x | v3.0.x | Supported by [Tyk OAS]({{< ref "api-management/gateway-config-tyk-oas#tyk-oas-api-definition-object" >}}) | @@ -408,7 +408,7 @@ An example is given below for illustrative purposes only. Tested Versions and Co | Third Party Dependency | Tested Versions | Compatible Versions | Comments | | ------------------------------------------------------------- | --------------- | ------------------- | ------------------------------------------------------------------------------------------- | -| [Go](https://go.dev/dl/) | 1.22 | 1.22 | [Go plugins]({{< ref "/plugins/supported-languages/golang" >}}) must be built using Go 1.22 | +| [Go](https://go.dev/dl/) | 1.22 | 1.22 | [Go plugins]({{< ref "api-management/plugins/golang#" >}}) must be built using Go 1.22 | | [Redis](https://redis.io/download/) | 6.2.x, 7.x | 6.2.x, 7.x | Used by Tyk Gateway | | [OpenAPI Specification](https://spec.openapis.org/oas/v3.0.3) | v3.0.x | v3.0.x | Supported by [Tyk OAS]({{< ref "api-management/gateway-config-tyk-oas#tyk-oas-api-definition-object" >}}) | @@ -590,7 +590,7 @@ An example is given below for illustrative purposes only. Tested Versions and Co | Third Party Dependency | Tested Versions | Compatible Versions | Comments | | ------------------------------------------------------------- | --------------- | ------------------- | ------------------------------------------------------------------------------------------- | -| [Go](https://go.dev/dl/) | 1.22 | 1.22 | [Go plugins]({{< ref "/plugins/supported-languages/golang" >}}) must be built using Go 1.22 | +| [Go](https://go.dev/dl/) | 1.22 | 1.22 | [Go plugins]({{< ref "api-management/plugins/golang#" >}}) must be built using Go 1.22 | | [Redis](https://redis.io/download/) | 6.2.x, 7.x | 6.2.x, 7.x | Used by Tyk Gateway | | [OpenAPI Specification](https://spec.openapis.org/oas/v3.0.3) | v3.0.x | v3.0.x | Supported by [Tyk OAS]({{< ref "api-management/gateway-config-tyk-oas#tyk-oas-api-definition-object" >}}) | @@ -815,7 +815,7 @@ There are no breaking changes in this release. | Third Party Dependency | Tested Versions | Compatible Versions | Comments | | ------------------------------------------------------------ | ---------------------- | ---------------------- | -------- | -| [Go](https://go.dev/dl/) | 1.21 | 1.21 | [Go plugins]({{< ref "/plugins/supported-languages/golang" >}}) must be built using Go 1.21 | +| [Go](https://go.dev/dl/) | 1.21 | 1.21 | [Go plugins]({{< ref "api-management/plugins/golang#" >}}) must be built using Go 1.21 | | [Redis](https://redis.io/download/) | 6.2.x, 7.x | 6.2.x, 7.x | Used by Tyk Gateway | | [OpenAPI Specification](https://spec.openapis.org/oas/v3.0.3)| v3.0.x | v3.0.x | Supported by [Tyk OAS]({{< ref "api-management/gateway-config-tyk-oas#tyk-oas-api-definition-object" >}}) | @@ -870,7 +870,7 @@ There are no breaking changes in this release. | Third Party Dependency | Tested Versions | Compatible Versions | Comments | | ------------------------------------------------------------ | ---------------------- | ---------------------- | -------- | -| [Go](https://go.dev/dl/) | 1.21 | 1.21 | [Go plugins]({{< ref "/plugins/supported-languages/golang" >}}) must be built using Go 1.21 | +| [Go](https://go.dev/dl/) | 1.21 | 1.21 | [Go plugins]({{< ref "api-management/plugins/golang#" >}}) must be built using Go 1.21 | | [Redis](https://redis.io/download/) | 6.2.x, 7.x | 6.2.x, 7.x | Used by Tyk Gateway | | [OpenAPI Specification](https://spec.openapis.org/oas/v3.0.3)| v3.0.x | v3.0.x | Supported by [Tyk OAS]({{< ref "api-management/gateway-config-tyk-oas#tyk-oas-api-definition-object" >}}) | @@ -997,7 +997,7 @@ An example is given below for illustrative purposes only. Tested Versions and Co | Third Party Dependency | Tested Versions | Compatible Versions | Comments | | ------------------------------------------------------------ | ---------------------- | ---------------------- | -------- | -| [Go](https://go.dev/dl/) | 1.21 | 1.21 | [Go plugins]({{< ref "/plugins/supported-languages/golang" >}}) must be built using Go 1.21 | +| [Go](https://go.dev/dl/) | 1.21 | 1.21 | [Go plugins]({{< ref "api-management/plugins/golang#" >}}) must be built using Go 1.21 | | [Redis](https://redis.io/download/) | 6.2.x, 7.x | 6.2.x, 7.x | Used by Tyk Gateway | | [OpenAPI Specification](https://spec.openapis.org/oas/v3.0.3)| v3.0.x | v3.0.x | Supported by [Tyk OAS]({{< ref "api-management/gateway-config-tyk-oas#tyk-oas-api-definition-object" >}}) | @@ -1188,7 +1188,7 @@ An example is given below for illustrative purposes only. Tested Versions and Co | Third Party Dependency | Tested Versions | Compatible Versions | Comments | | ------------------------------------------------------------ | ---------------------- | ---------------------- | -------- | -| [Go](https://go.dev/dl/) | 1.19 (GQL), 1.21 (GW) | 1.19 (GQL), 1.21 (GW) | [Go plugins]({{< ref "/plugins/supported-languages/golang" >}}) must be built using Go 1.21 | +| [Go](https://go.dev/dl/) | 1.19 (GQL), 1.21 (GW) | 1.19 (GQL), 1.21 (GW) | [Go plugins]({{< ref "api-management/plugins/golang#" >}}) must be built using Go 1.21 | | [Redis](https://redis.io/download/) | 6.2.x, 7.x | 6.2.x, 7.x | Used by Tyk Gateway | | [OpenAPI Specification](https://spec.openapis.org/oas/v3.0.3)| v3.0.x | v3.0.x | Supported by [Tyk OAS]({{< ref "api-management/gateway-config-tyk-oas#tyk-oas-api-definition-object" >}}) | @@ -1508,7 +1508,7 @@ This release has no breaking changes. | Third Party Dependency | Tested Versions | Compatible Versions | Comments | | ------------------------------------------------------------- | --------------------- | --------------------- | ------------------------------------------------------------------------------------------ | -| [Go](https://go.dev/dl/) | 1.22 (GW) | 1.22 (GW) | [Go plugins]({{< ref "plugins/supported-languages/golang" >}}) must be built using Go 1.22 | +| [Go](https://go.dev/dl/) | 1.22 (GW) | 1.22 (GW) | [Go plugins]({{< ref "api-management/plugins/golang#" >}}) must be built using Go 1.22 | | [Redis](https://redis.io/download/) | 6.2.x, 7.x | 6.2.x, 7.x | Used by Tyk Gateway | | [OpenAPI Specification](https://spec.openapis.org/oas/v3.0.3) | v3.0.x | v3.0.x | Supported by [Tyk OAS]({{< ref "api-management/gateway-config-tyk-oas#tyk-oas-api-definition-object" >}}) | @@ -1669,7 +1669,7 @@ An example is given below for illustrative purposes only. Tested Versions and Co | Third Party Dependency | Tested Versions | Compatible Versions | Comments | | ------------------------------------------------------------- | --------------------- | --------------------- | ------------------------------------------------------------------------------------------ | -| [Go](https://go.dev/dl/) | 1.22 (GW) | 1.22 (GW) | [Go plugins]({{< ref "plugins/supported-languages/golang" >}}) must be built using Go 1.22 | +| [Go](https://go.dev/dl/) | 1.22 (GW) | 1.22 (GW) | [Go plugins]({{< ref "api-management/plugins/golang#" >}}) must be built using Go 1.22 | | [Redis](https://redis.io/download/) | 6.2.x, 7.x | 6.2.x, 7.x | Used by Tyk Gateway | | [OpenAPI Specification](https://spec.openapis.org/oas/v3.0.3) | v3.0.x | v3.0.x | Supported by [Tyk OAS]({{< ref "api-management/gateway-config-tyk-oas#tyk-oas-api-definition-object" >}}) | @@ -1802,7 +1802,7 @@ An example is given below for illustrative purposes only. Tested Versions and Co | Third Party Dependency | Tested Versions | Compatible Versions | Comments | | ------------------------------------------------------------- | --------------- | ------------------- | ------------------------------------------------------------------------------------------ | -| [Go](https://go.dev/dl/) | 1.22 | 1.22 | [Go plugins]({{< ref "plugins/supported-languages/golang" >}}) must be built using Go 1.22 | +| [Go](https://go.dev/dl/) | 1.22 | 1.22 | [Go plugins]({{< ref "api-management/plugins/golang#" >}}) must be built using Go 1.22 | | [Redis](https://redis.io/download/) | 6.2.x, 7.x | 6.2.x, 7.x | Used by Tyk Gateway | | [OpenAPI Specification](https://spec.openapis.org/oas/v3.0.3) | v3.0.x | v3.0.x | Supported by [Tyk OAS]({{< ref "api-management/gateway-config-tyk-oas#tyk-oas-api-definition-object" >}}) | @@ -1924,7 +1924,7 @@ An example is given below for illustrative purposes only. Tested Versions and Co | Third Party Dependency | Tested Versions | Compatible Versions | Comments | | ------------------------------------------------------------- | --------------- | ------------------- | ------------------------------------------------------------------------------------------ | -| [Go](https://go.dev/dl/) | 1.22 | 1.22 | [Go plugins]({{< ref "plugins/supported-languages/golang" >}}) must be built using Go 1.22 | +| [Go](https://go.dev/dl/) | 1.22 | 1.22 | [Go plugins]({{< ref "api-management/plugins/golang#" >}}) must be built using Go 1.22 | | [Redis](https://redis.io/download/) | 6.2.x, 7.x | 6.2.x, 7.x | Used by Tyk Gateway | | [OpenAPI Specification](https://spec.openapis.org/oas/v3.0.3) | v3.0.x | v3.0.x | Supported by [Tyk OAS]({{< ref "api-management/gateway-config-tyk-oas#tyk-oas-api-definition-object" >}}) | @@ -1990,7 +1990,7 @@ attack surface by eliminating unnecessary packages, which bolsters the security
Custom Response Plugins not working for Tyk OAS APIs -We have resolved an issue where custom [response plugins]({{< ref "plugins/plugin-types/response-plugins" >}}) were not being +We have resolved an issue where custom [response plugins]({{< ref "api-management/plugins/plugin-types#response-plugins" >}}) were not being triggered for Tyk OAS APIs. This fix ensures that all [supported]({{< ref "api-management/gateway-config-tyk-oas#tyk-oas-api-feature-status" >}}) custom plugins are invoked as expected when using Tyk OAS APIs. @@ -2116,7 +2116,7 @@ An example is given below for illustrative purposes only. Tested Versions and Co | Third Party Dependency | Tested Versions | Compatible Versions | Comments | | ------------------------------------------------------------- | --------------------- | --------------------- | ------------------------------------------------------------------------------------------ | -| [Go](https://go.dev/dl/) | 1.19 (GQL), 1.21 (GW) | 1.19 (GQL), 1.21 (GW) | [Go plugins]({{< ref "plugins/supported-languages/golang" >}}) must be built using Go 1.21 | +| [Go](https://go.dev/dl/) | 1.19 (GQL), 1.21 (GW) | 1.19 (GQL), 1.21 (GW) | [Go plugins]({{< ref "api-management/plugins/golang#" >}}) must be built using Go 1.21 | | [Redis](https://redis.io/download/) | 6.2.x, 7.x | 6.2.x, 7.x | Used by Tyk Gateway | | [OpenAPI Specification](https://spec.openapis.org/oas/v3.0.3) | v3.0.x | v3.0.x | Supported by [Tyk OAS]({{< ref "api-management/gateway-config-tyk-oas#tyk-oas-api-definition-object" >}}) | @@ -2257,7 +2257,7 @@ An example is given below for illustrative purposes only. Tested Versions and Co | Third Party Dependency | Tested Versions | Compatible Versions | Comments | | ------------------------------------------------------------- | --------------------- | --------------------- | ------------------------------------------------------------------------------------------ | -| [Go](https://go.dev/dl/) | 1.19 (GQL), 1.21 (GW) | 1.19 (GQL), 1.21 (GW) | [Go plugins]({{< ref "plugins/supported-languages/golang" >}}) must be built using Go 1.21 | +| [Go](https://go.dev/dl/) | 1.19 (GQL), 1.21 (GW) | 1.19 (GQL), 1.21 (GW) | [Go plugins]({{< ref "api-management/plugins/golang#" >}}) must be built using Go 1.21 | | [Redis](https://redis.io/download/) | 6.2.x, 7.x | 6.2.x, 7.x | Used by Tyk Gateway | | [OpenAPI Specification](https://spec.openapis.org/oas/v3.0.3) | v3.0.x | v3.0.x | Supported by [Tyk OAS]({{< ref "api-management/gateway-config-tyk-oas#tyk-oas-api-definition-object" >}}) | @@ -2343,7 +2343,7 @@ An example is given below for illustrative purposes only. Tested Versions and Co | Third Party Dependency | Tested Versions | Compatible Versions | Comments | | ------------------------------------------------------------- | --------------------- | --------------------- | ------------------------------------------------------------------------------------------ | -| [Go](https://go.dev/dl/) | 1.19 (GQL), 1.21 (GW) | 1.19 (GQL), 1.21 (GW) | [Go plugins]({{< ref "plugins/supported-languages/golang" >}}) must be built using Go 1.21 | +| [Go](https://go.dev/dl/) | 1.19 (GQL), 1.21 (GW) | 1.19 (GQL), 1.21 (GW) | [Go plugins]({{< ref "api-management/plugins/golang#" >}}) must be built using Go 1.21 | | [Redis](https://redis.io/download/) | 6.2.x, 7.x | 6.2.x, 7.x | Used by Tyk Gateway | | [OpenAPI Specification](https://spec.openapis.org/oas/v3.0.3) | v3.0.x | v3.0.x | Supported by [Tyk OAS]({{< ref "api-management/gateway-config-tyk-oas#tyk-oas-api-definition-object" >}}) | @@ -2500,7 +2500,7 @@ An example is given below for illustrative purposes only. Tested Versions and Co | Third Party Dependency | Tested Versions | Compatible Versions | Comments | | ------------------------------------------------------------- | --------------------- | --------------------- | ------------------------------------------------------------------------------------------ | -| [Go](https://go.dev/dl/) | 1.19 (GQL), 1.21 (GW) | 1.19 (GQL), 1.21 (GW) | [Go plugins]({{< ref "plugins/supported-languages/golang" >}}) must be built using Go 1.21 | +| [Go](https://go.dev/dl/) | 1.19 (GQL), 1.21 (GW) | 1.19 (GQL), 1.21 (GW) | [Go plugins]({{< ref "api-management/plugins/golang#" >}}) must be built using Go 1.21 | | [Redis](https://redis.io/download/) | 6.2.x, 7.x | 6.2.x, 7.x | Used by Tyk Gateway | | [OpenAPI Specification](https://spec.openapis.org/oas/v3.0.3) | v3.0.x | v3.0.x | Supported by [Tyk OAS]({{< ref "api-management/gateway-config-tyk-oas#tyk-oas-api-definition-object" >}}) | @@ -2644,7 +2644,7 @@ An example is given below for illustrative purposes only. Tested Versions and Co | Third Party Dependency | Tested Versions | Compatible Versions | Comments | | ------------------------------------------------------------- | --------------------- | --------------------- | ------------------------------------------------------------------------------------------ | -| [Go](https://go.dev/dl/) | 1.19 (GQL), 1.21 (GW) | 1.19 (GQL), 1.21 (GW) | [Go plugins]({{< ref "plugins/supported-languages/golang" >}}) must be built using Go 1.21 | +| [Go](https://go.dev/dl/) | 1.19 (GQL), 1.21 (GW) | 1.19 (GQL), 1.21 (GW) | [Go plugins]({{< ref "api-management/plugins/golang#" >}}) must be built using Go 1.21 | | [Redis](https://redis.io/download/) | 6.2.x, 7.x | 6.2.x, 7.x | Used by Tyk Gateway | | [OpenAPI Specification](https://spec.openapis.org/oas/v3.0.3) | v3.0.x | v3.0.x | Supported by [Tyk OAS]({{< ref "api-management/gateway-config-tyk-oas#tyk-oas-api-definition-object" >}}) | @@ -2828,7 +2828,7 @@ Starting from Tyk Gateway version v5.3.0, Python is no longer bundled with the o reduce exposure to security vulnerabilities in the Python libraries. Whilst the Gateway still supports Python plugins, you must [extend -the image]({{< ref "plugins/supported-languages/rich-plugins/python/python#install-the-python-development-packages" >}}) +the image]({{< ref "api-management/plugins/rich-plugins#install-the-python-development-packages" >}}) to add the language support. - -Welcome to the Tyk Plugins Hub, dedicated to providing you with a curated list of resources that showcase how to develop Tyk Plugins. - -[Tyk Plugins]({{< ref "plugins" >}}) are a powerful tool that allows you to develop custom middleware that can intercept requests at different stages of the request lifecycle, modifying/transforming headers and body content. - -Tyk has extensive support for writing custom plugins using a wide range of languages, most notably: Go, Python, Javascript etc. In fact, plugins can be developed using most languages via *gRPC*. - -## Blogs - -Selected blogs for plugin development are included below. Further examples are available at the Tyk [website](https://tyk.io/?s=plugin). - -#### 1. [Decoupling micro-services using Message-based RPC](https://medium.com/@asoorm/decoupling-micro-services-using-message-based-rpc-fa1c12409d8f) -- **Summary**: Explains how to write a plugin that intercepts an API request and forwards it to a gRPC server. The gRPC server processes the request and dispatches work to an RabbitMQ message queue. The source code is available in the accompanying [GitHub repository](https://github.com/asoorm/tyk-rmq-middleware) - -#### 2. [How to configure a gRPC server using Tyk](https://tyk.io/blog/how-to-configure-a-grpc-server-using-tyk/) -- **Summary**: Explains how to configure a Python implementation of a gRPC server to add additional logic to API requests. During the request lifecycle, the Tyk-Gateway acts as a gRPC client that contacts the Python gRPC server, providing additional custom logic. - -#### 3. [How to deploy Python plugins in Tyk running On Kubernetes](https://tyk.io/blog/how-to-deploy-python-plugins-in-tyk-running-on-kubernetes/) -- **Summary**: Explains how to deploy a custom Python plugin into a Tyk installation running on a Kubernetes cluster. - -## GitHub Repositories - -Here are some carefully selected GitHub repositories that will help you learn how to integrate and utilize Tyk Plugins in your development projects: - -#### 1. [Tyk Awesome Plugins](https://github.com/TykTechnologies/tyk-awesome-plugins) -- **Description**: Index of plugins developed using a variety of languages. -- **Key Features Demonstrated**: A comprehensive index for a collection of plugins that can be used with the Tyk API Gateway in areas such as: rate limiting, authentication and request transformation. The examples are developed using a diverse array of languages, including but not limited to: Python, JavaScript and Go. This broad language support ensures that developers from different backgrounds and with various language preferences can seamlessly integrate these plugins with their Tyk API Gateway implementations. - -#### 2. [Custom Plugin Examples](https://github.com/TykTechnologies/custom-plugin-examples/tree/master) -- **Description**: Index of examples for a range of plugin hooks (Pre, Post, Post-Auth and Response) developed using a variety of languages. -- **Key Features Demonstrated**: Specific examples include invoking an AWS lambda, inserting a new claim into a JWT, inject a signed JWT into authorization header, request header modification. A range of examples are available including Python, Java, Ruby, Javascript, NodeJS and Go. - -#### 3. [Environment For Plugin Development](https://github.com/TykTechnologies/custom-go-plugin) -- **Description**: Provides a docker-compose environment for developing your own custom Go plugins. -- **Key Features Demonstrated**: Showcases support for bundling plugins, uploading plugins to AWS S3 storage, test coverage etc. - -## Conclusion - -[Tyk Plugins]({{< ref "plugins" >}}) are a valuable asset in the Tyk API Platform. Plugins enable you to develop custom middleware for intercepted upstream API requests and responses. The resources included in this Tyk Plugin Hub serve to help you harness the capabilities and use cases for Tyk Plugins. By exploring these resources and experimenting with the code, you can gain a deeper understanding of how to leverage [Tyk Plugins]({{< ref "plugins" >}}) to build innovative and efficient solutions for your projects. - -Happy coding! - - diff --git a/tyk-docs/content/plugins/plugin-types/analytics-plugins.md b/tyk-docs/content/plugins/plugin-types/analytics-plugins.md deleted file mode 100755 index f5c91b7b43..0000000000 --- a/tyk-docs/content/plugins/plugin-types/analytics-plugins.md +++ /dev/null @@ -1,80 +0,0 @@ ---- -date: 2022-07-25 -title: Analytics Plugins -menu: - main: - parent: "Plugin Types" -weight: 90 -aliases: - - /plugins/analytics-plugins ---- - -Since Tyk 4.1.0 we have incorporated analytic plugins which enables editing or removal of all parts of analytics records and raw request and responses recorded by Tyk at the gateway level. This feature leverages existing Go plugin infrastructure. - -- Tyk receives the request. -- Tyk runs the full middleware chain, including any other plugins hooks like Pre, Post, Custom Authentication, etc. -- Tyk sends the request to your upstream API. -- The response is received and analytics plugin function is triggered before recording the hit to redis. -- Your plugin modifies the analytics record and sends it back to Tyk. -- Tyk takes the modified analytics record and record the hit in redis. - -Example analytics Go plugins can be found [here](https://github.com/TykTechnologies/tyk/blob/master/test/goplugins/test_goplugin.go#L149) - ---- - -An analytics plugin is configured using the `analytics_plugin` configuration block within an API Definition. This contains the following configuration parameters: - -- `enable`: Set to `true` to enable the plugin -- `func_name`: The name of the function representing the plugin -- `plugin_path`: The path to the source code file containing the function that implements the plugin - -{{< tabs_start >}} - -{{< tab_start "Tyk Gateway" >}} - -To enable the analytics rewriting functionality, adjust the following in API definition: - -```json -{ - "analytics_plugin": { - "enable": true, - "func_name": "", - "plugin_path": "/analytics_plugin.so" - } -} -``` - -{{< tab_end >}} - -{{< tab_start "Tyk Operator" >}} - -The example API Definition resource listed below listens on path */httpbin* and forwards requests upstream to *http://httpbin.org*. A Go Analytics Plugin is enabled for function *MaskAnalyticsData*, located within the */opt/tyk-gateway/plugins/example-plugin.so* shared object file. - -```yaml {linenos=table,hl_lines=["15-18"],linenostart=1} -apiVersion: tyk.tyk.io/v1alpha1 -kind: ApiDefinition -metadata: - name: analytics-plugin -spec: - name: httpbin-analytics-plugin - active: true - protocol: http - proxy: - listen_path: /httpbin - strip_listen_path: true - target_url: http://httpbin.org - use_keyless: true - enable_detailed_recording: true - analytics_plugin: - enable: true - func_name: MaskAnalyticsData # Replace it with function name of your plugin - plugin_path: /opt/tyk-gateway/plugins/example-plugin.so # Replace it with path of your plugin file -``` - -{{< tab_end >}} - -{{< tabs_end >}} - ---- - -
diff --git a/tyk-docs/content/plugins/plugin-types/auth-plugins/auth-plugins.md b/tyk-docs/content/plugins/plugin-types/auth-plugins/auth-plugins.md deleted file mode 100755 index 6b86f40782..0000000000 --- a/tyk-docs/content/plugins/plugin-types/auth-plugins/auth-plugins.md +++ /dev/null @@ -1,71 +0,0 @@ ---- -date: 2017-03-24T15:45:13Z -title: Authentication Plugins -tags: ["Custom Auth Plugins", "Custom Auth", "Auth Plugins", "Authentication Plugins"] -menu: - main: - parent: "Plugin Types" -weight: 11 -aliases: - - "/plugins/auth-plugins" - ---- - -If you have unique authentication requirements, you can write a custom authentication plugin. - -## Session Authentication and Authorization - -A very important thing to understand when using custom authentication plugins is that Tyk will continue to perform session authentication and authorization using the information returned by your plugin. Tyk will cache this Session information. **This is necessary in order to do things like rate limiting, access control, quotas, throttling, etc.** - -Tyk will try to be clever about what to cache, but we need to help it. There are two ways to do that, with and without the `ID Extractor`. - -### The ID Extractor - -The ID Extractor is a caching mechanism that's used in combination with Tyk Plugins. It can be used specifically with plugins that implement custom authentication mechanisms. The ID Extractor works for all rich plugins: gRPC-based plugins, Python and Lua. - -See [ID Extractor]({{< ref "plugins/plugin-types/auth-plugins/id-extractor" >}}) for more details. - -### Token Metadata - -Tyk creates an in-memory object to track the rate limit, quotas, and more for each session. - -This is why we set the `token` metadata when using custom authentication middleware, in order to give Tyk a unique ID with which to track each session. - -For backwards compatibility, even when using an ID Extractor, we need to continue to set the `token` metadata. For example, when building a session object in GoLang custom middleware: - -```{.copyWrapper} -object.Session = &coprocess.SessionState{ - LastUpdated: time.Now().String(), - Rate: 5, - Per: 10, - QuotaMax: int64(0), - QuotaRenews: time.Now().Unix(), - IdExtractorDeadline: extractorDeadline, - Metadata: map[string]string{ - "token": "my-unique-token", - }, - ApplyPolicies: ["5d8929d8f56e1a138f628269"], - } -``` -[source](https://github.com/TykTechnologies/tyk-grpc-go-basicauth-jwt/blob/master/main.go#L102) - -### Without ID Extractor - -When not using ID Extractor, Tyk will continue to cache authenticated sessions returned by custom auth plugins. We must set a unique `token` field in the Metadata (see above) that Tyk will use to cache. - -## Supported Languages - -The following languages are supported for custom authentication plugins: - -- All Rich Plugins (gRPC, Python, Lua) -- GoLang - -See the [supported languages]({{< ref "plugins/supported-languages" >}}) section for custom authentication plugin examples in a language of your choosing. There's also a [blog that walks you through setting up gRPC custom auth in Java](https://tyk.io/how-to-setup-custom-authentication-middleware-using-grpc-and-java/). - -## Tyk Operator - -Please consult the Tyk Operator supporting documentation for examples of how to configure a Tyk Operator API to use: - -- [Go custom authentication plugin]({{< ref "api-management/automations/operator#custom-plugin-auth-go" >}}) -- [gRPC custom authentication plugin]({{< ref "api-management/automations/operator#custom-plugin-auth-grpc" >}}) - diff --git a/tyk-docs/content/plugins/plugin-types/auth-plugins/id-extractor.md b/tyk-docs/content/plugins/plugin-types/auth-plugins/id-extractor.md deleted file mode 100644 index 20831eaf53..0000000000 --- a/tyk-docs/content/plugins/plugin-types/auth-plugins/id-extractor.md +++ /dev/null @@ -1,196 +0,0 @@ ---- -date: 2017-03-24T13:04:21Z -title: ID Extractor - Plugin Caching Mechanism -description: "This page describes the caching mechanism Tyk Gateway provides when using plugins as one of the middlewares in the API request execution chain." -tags: ["ID Extractor", "Plugin caching", "Tyk plugin"] -menu: - main: - parent: "Authentication Plugins" -weight: 3 -aliases: - - /customise-tyk/plugins/rich-plugins/id-extractor/ - - /plugins/rich-plugins/id-extractor - - /plugins/auth-plugins/id-extractor ---- - -## Introduction - -The ID extractor is a caching mechanism that's used in combination with Tyk Plugins. It is used specifically with plugins that implement custom authentication mechanisms. - -We use the term "ID" to describe any key that's used for authentication purposes. - -When a custom authentication mechanism is used, every API call triggers a call to the associated middleware function, if you're using a gRPC-based plugin this translates into a gRPC call. If you're using a native plugin -like a Python plugin-, this involves a Python interpreter call. - -The ID extractor works the following rich plugins: gRPC-based plugins, Python and Lua. - -## When to use the ID Extractor? - -The main idea of the ID extractor is to reduce the number of calls made to your plugin and cache the API keys that have been already authorized by your authentication mechanism. This means that after a successful authentication event, subsequent calls will be handled by the Tyk Gateway and its Redis cache, resulting in a performance similar to the built-in authentication mechanisms that Tyk provides. - -## When does the ID Extractor Run? - -When enabled, the ID extractor runs right before the authentication step, allowing it to take control of the flow and decide whether to call your authentication mechanism or not. - -If my ID is cached by this mechanism and my plugin isn't longer called, how do I expire it? -When you implement your own authentication mechanism using plugins, you initialise the session object from your own code. The session object has a field that's used to configure the lifetime of a cached ID, this field is called `id_extractor_deadline`. See [Plugin Data Structures]({{< ref "plugins/supported-languages/rich-plugins/rich-plugins-data-structures" >}}) for more details. -The value of this field should be a UNIX timestamp on which the cached ID will expire, like `1507268142958`. It's an integer. - -For example, this snippet is used in a NodeJS plugin, inside a custom authentication function: - -``` -// Initialize a session state object - var session = new tyk.SessionState() - // Get the current UNIX timestamp - var timestamp = Math.floor( new Date() / 1000 ) - // Based on the current timestamp, add 60 seconds: - session.id_extractor_deadline = timestamp + 60 - // Finally inject our session object into the request object: - Obj.session = session -``` - -If you already have a plugin that implements a custom authentication mechanism, appending the `id_extractor_deadline` and setting its value is enough to activate this feature. -In the above sample, Tyk will cache the key for 60 seconds. During that time any requests that use the cached ID won't call your plugin. - -## How to enable the ID Extractor - -The ID extractor is configured on a per API basis. -The API should be a protected one and have the `enable_coprocess_auth` flag set to true, like the following definition: - -```json -{ - "name": "Test API", - "api_id": "my-api", - "org_id": "my-org", - "use_keyless": false, - "auth": { - "auth_header_name": "Authorization" - }, - "proxy": { - "listen_path": "/test-api/", - "target_url": "http://httpbin.org/", - "strip_listen_path": true - }, - "enable_coprocess_auth": true, - "custom_middleware_bundle": "bundle.zip" -} -``` - -If you're not using the Community Edition, check the API settings in the dashboard and make sure that "Custom Auth" is selected. - -The second requirement is to append an additional configuration block to your plugin manifest file, using the `id_extractor` key: - -```json -{ - "custom_middleware": { - "auth_check": { "name": "MyAuthCheck" }, - "id_extractor": { - "extract_from": "header", - "extract_with": "value", - "extractor_config": { - "header_name": "Authorization" - } - }, - "driver": "grpc" - } -} -``` - -* `extract_from` specifies the source of the ID to extract. -* `extract_with` specifies how to extract and parse the extracted ID. -* `extractor_config` specifies additional parameters like the header name or the regular expression to use, this is different for every choice, see below for more details. - - -## Available ID Extractor Sources - -### Header Source - -Use this source to extract the key from a HTTP header. Only the name of the header is required: - -```json -{ - "id_extractor": { - "extract_from": "header", - "extract_with": "value", - "extractor_config": { - "header_name": "Authorization" - } - } -} -``` - -### Form source - -Use this source to extract the key from a submitted form, where `param_name` represents the key of the submitted parameter: - - -```json -{ - "id_extractor": { - "extract_from": "form", - "extract_with": "value", - "extractor_config": { - "param_name": "my_param" - } - } -} -``` - - -## Available ID Extractor Modes - -### Value Extractor - -Use this to take the value as its present. This is commonly used in combination with the header source: - -```json -{ - "id_extractor": { - "extract_from": "header", - "extract_with": "value", - "extractor_config": { - "header_name": "Authorization" - } - } -} -``` - -### Regular Expression Extractor - -Use this to match the ID with a regular expression. This requires additional parameters like `regex_expression`, which represents the regular expression itself and `regex_match_index` which is the item index: - -```json -{ - "id_extractor": { - "extract_from": "header", - "extract_with": "regex", - "extractor_config": { - "header_name": "Authorization", - "regex_expression": "[^-]+$", - "regex_match_index": 0 - } - } -} -``` - -Using the example above, if we send a header like `prefix-d28e17f7`, given the regular expression we're using, the extracted ID value will be `d28e17f7`. - -## Example Session -Here's an example of a Session being built in GoLang custom middleware: -```{.copyWrapper} -extractorDeadline := time.Now().Add(time.Second * 5).Unix() -object.Session = &coprocess.SessionState{ - - LastUpdated: time.Now().String(), - Rate: 5, - Per: 10, - QuotaMax: int64(0), - QuotaRenews: time.Now().Unix(), - Metadata: map[string]string{ - "token": "my-unique-token", - }, - ApplyPolicies: ["5d8929d8f56e1a138f628269"], - } -``` -[source](https://github.com/TykTechnologies/tyk-grpc-go-basicauth-jwt/blob/master/main.go#L102) - -Note: When using an ID Extractor, you must set a `LastUpdated` or else token updates will not be applied. If you don't set an ID Extractor, Tyk will store session information in the cache based off the `token` field that is set in the metadata. diff --git a/tyk-docs/content/plugins/plugin-types/plugintypes.md b/tyk-docs/content/plugins/plugin-types/plugintypes.md deleted file mode 100644 index 0c1f7d0544..0000000000 --- a/tyk-docs/content/plugins/plugin-types/plugintypes.md +++ /dev/null @@ -1,27 +0,0 @@ ---- -title: "Plugin Types" -menu: - main: - parent: "Custom Plugins" -weight: 10 ---- - -Custom Plugins enable users to execute custom code to complete tasks specific to their particular use case. This allows users to complete tasks that would not otherwise be possible using Tyk's standard middleware options. Tyk has a [pre-defined execution order]({{< ref "/concepts/middleware-execution-order" >}}) for the middleware which also includes seven hooks for the custom plugins. As such, users can execute, or "hook", their plugin in these phases of the API request/response lifecycle based on their specific use case. - -## Plugin and Hook Types -This table includes all the plugin types with the relevant hooks, their place in the execution chain, description and examples: - -| Hook Type (in their execution order) | Plugin Type | HTTP Request/Response phase | Executed before/after reverse proxy to the upstream API | Details | Common Use Cases | -|--------------------------|----|---|--------------|--------------------|--------- -| Pre (Request) | Request Plugin | HTTP request | Before | The first thing to be executed, before any middleware | IP Rate Limit plugins, API Request enrichment | -| Authentication| Authentication Plugin | HTTP request | Before | Replaces Tyk's authentication & authorization middleware with your own business logic | When you need your a custom flow, for example, interfacing with legacy Auth database | -| Post-Auth (Request)| Authentication Plugin | HTTP request | Before | Executed immediately after authentication middleware | Additional special custom authentication is needed | -| Post (Request)| Request Plugin | HTTP request| Before | The final middleware to be executed during the *HTTP request* phase (see **Note** below) | Update the request before it gets to the upstream, for example, adding a header that might override another header, so we add it at the end to ensure it doesn't get overridden | -| Response Plugin| Response Plugin | HTTP Response | After | Executed after the reverse proxy to the upstream API | Executed straight after the reverse proxy returns from the upstream API to Tyk | Change the response before the user gets it, for example, change `Location` header from internal to an external URL | -| Analytics Plugin (Request+Response)| Analytics Plugin | HTTP request | After | The final middleware to be executed during the *HTTP response* phase | Change analytics records, for example, obfuscating sensitive data such as the `Authorization` header | - -{{< note success >}} -**Note** - -There are two different options for the Post Plugin that is executed at the end of the request processing chain. The API-level Post Plugin is applied to all requests, whilst the [endpoint-level]({{< ref "product-stack/tyk-gateway/middleware/endpoint-plugin" >}}) custom Golang plugin is only applied to requests made to specific endpoints. If both are configured, the endpoint-level plugin will be executed first. -{{< /note >}} diff --git a/tyk-docs/content/plugins/plugin-types/request-plugins.md b/tyk-docs/content/plugins/plugin-types/request-plugins.md deleted file mode 100755 index 856d489b89..0000000000 --- a/tyk-docs/content/plugins/plugin-types/request-plugins.md +++ /dev/null @@ -1,63 +0,0 @@ ---- -date: 2017-03-24T15:45:13Z -title: Request Plugins -menu: - main: - parent: "Plugin Types" -weight: 10 -aliases: - - /plugins/request-plugins ---- - -There are 4 different phases in the [request lifecycle]({{< ref "concepts/middleware-execution-order" >}}) you can inject custom plugins, including [Authentication plugins]({{< ref "plugins/plugin-types/auth-plugins/auth-plugins" >}}). There are performance advantages to picking the correct phase, and of course that depends on your use case and what functionality you need. - -### Hook Capabilities -| Functionality | Pre | Auth | Post-Auth | Post | -|-------------------------|----------|-------------|-----------|-----------| -| Can modify the Header | ✅ | ✅ | ✅ | ✅ -| Can modify the Body | ✅ | ✅ | ✅ |✅ -| Can modify Query Params | ✅ | ✅ | ✅ |✅ -| Can view Session1 Details (metadata, quota, context-vars, tags, etc) | ❌ | ✅ |✅ |✅ -| Can modify Session1 2 | ❌ | ✅ | ❌ |❌ -| Can Add More Than One3 | ✅ | ❌ |✅ | ✅ - -[1] A [Session object]({{< ref "getting-started/key-concepts/what-is-a-session-object" >}}) contains allowances and identity information that is unique to each requestor - -[2] You can modify the session by using your programming language's SDK for Redis. Here is an [example](https://github.com/TykTechnologies/custom-plugins/blob/master/plugins/go-auth-multiple_hook_example/main.go#L135) of doing that in Golang. - -[3] For select hook locations, you can add more than one plugin. For example, in the same API request, you can have 3 Pre, 1 auth, 5 post-auth, and 2 post plugins. - -### Return Overrides / ReturnOverrides -You can have your plugin finish the request lifecycle and return a response with custom payload & headers to the requestor. - -[Read more here]({{< ref "plugins/supported-languages/rich-plugins/rich-plugins-data-structures#returnoverrides" >}}) - -##### Python Example - -```python -from tyk.decorators import * - -@Hook -def MyCustomMiddleware(request, session, spec): - print("my_middleware: MyCustomMiddleware") - request.object.return_overrides.headers['content-type'] = 'application/json' - request.object.return_overrides.response_code = 200 - request.object.return_overrides.response_error = "{\"key\": \"value\"}\n" - return request, session -``` - -##### JavaScript Example -```javascript -var testJSVMData = new TykJS.TykMiddleware.NewMiddleware({}); - -testJSVMData.NewProcessRequest(function(request, session, config) { - request.ReturnOverrides.ResponseError = "Foobarbaz" - request.ReturnOverrides.ResponseBody = "Foobar" - request.ReturnOverrides.ResponseCode = 200 - request.ReturnOverrides.ResponseHeaders = { - "X-Foo": "Bar", - "X-Baz": "Qux" - } - return testJSVMData.ReturnData(request, {}); -}); -``` diff --git a/tyk-docs/content/plugins/plugin-types/response-plugins.md b/tyk-docs/content/plugins/plugin-types/response-plugins.md deleted file mode 100755 index b079ee7b00..0000000000 --- a/tyk-docs/content/plugins/plugin-types/response-plugins.md +++ /dev/null @@ -1,94 +0,0 @@ ---- -date: 2017-03-24T15:45:13Z -title: Response Plugins -menu: - main: - parent: "Plugin Types" -weight: 20 -aliases: - - plugins/response-plugins ---- - -Since Tyk 3.0 we have incorporated response hooks, this type of hook allows you to modify the response object returned by the upstream. The flow is follows: - -- Tyk receives the request. -- Tyk runs the full middleware chain, including any other plugins hooks like Pre, Post, Custom Authentication, etc. -- Tyk sends the request to your upstream API. -- The request is received by Tyk and the response hook is triggered. -- Your plugin modifies the response and sends it back to Tyk. -- Tyk takes the modified response and is received by the client. - -This snippet illustrates the hook function signature: - -```python -@Hook -def ResponseHook(request, response, session, metadata, spec): - tyk.log("ResponseHook is called", "info") - # In this hook we have access to the response object, to inspect it, uncomment the following line: - # print(response) - tyk.log("ResponseHook: upstream returned {0}".format(response.status_code), "info") - # Attach a new response header: - response.headers["injectedkey"] = "injectedvalue" - return response -``` - -If working with a Tyk Classic API, you would add this configuration to the API definition: - -``` -{ - "custom_middleware": { - "response": [ - { - "name": "ResponseHook", - "path": "middleware/middleware.py" - } - ], - "driver": "python" - } -} -``` - - - `driver`: set this to the appropriate value for the plugin type (e.g. `python`, `goplugin`) - - `response`: this is the hook name. You use middleware with the `response` hook type because you want this custom middleware to process the request on its return leg of a round trip. - - `response.name`: is your function name from the plugin file. - - `response.path`: is the full or relative (to the Tyk binary) path to the plugin source file. Ensure Tyk has read access to this file. - -Starting from versions 5.0.4 and 5.1.1+ for our Go, Python and Ruby users we have introduced the `multivalue_headers` field to facilitate more flexible and efficient management of headers, particularly for scenarios involving a single header key associated with multiple values. The `multivalue_headers` field, similar to its predecessor, the `headers` field, is a key-value store. However, it can accommodate an array or list of string values for each key, instead of a single string value. This feature empowers you to represent multiple values for a single header key. Here's an example of how you might use `multivalue_headers`, using the Set-Cookie header which often has multiple values: - -``` -multivalue_headers = { - "Set-Cookie": ["sessionToken=abc123; HttpOnly; Secure", "language=en-US; Secure"], -} -``` - -In this example, Set-Cookie header has two associated values: `"sessionToken=abc123; HttpOnly; Secure"` and `"language=en-US; Secure"`. To help you understand this further, let's see how `multivalue_headers` can be used in a Tyk response plugin written in Python: - -```python -from tyk.decorators import * -from gateway import TykGateway as tyk - -@Hook -def Del_ResponseHeader_Middleware(request, response, session, metadata, spec): - # inject a new header with 2 values - new_header = response.multivalue_headers.add() - new_header.key = "Set-Cookie" - new_header.values.extend("sessionToken=abc123; HttpOnly; Secure") - new_header.values.extend("language=en-US; Secure") - - tyk.log(f"Headers content :\n {response.headers}\n----------", "info") - tyk.log(f"Multivalue Headers updated :\n {response.multivalue_headers}\n----------", "info") - - return response -``` - -In this script, we add 2 values for the `Set-Cookie` header and then log both: the traditional `headers` and the new `multivalue_headers`. This is a great way to monitor your transition to `multivalue_headers` and ensure that everything is functioning as expected. - -Please note, while the `headers` field will continue to be available and maintained for backward compatibility, we highly encourage the adoption of `multivalue_headers` for the added flexibility in handling multiple header values. - -### Go response plugins - -[Go response plugins]({{< ref "product-stack/tyk-gateway/advanced-configurations/plugins/golang/writing-go-plugins#creating-a-custom-response-plugin" >}}) have been available since Tyk v3.2. - -### Supported Response Plugin Languages - -See [Supported Plugins]({{< ref "plugins/supported-languages" >}}) for details on which languages the response plugin is supported in. diff --git a/tyk-docs/content/plugins/supported-languages.md b/tyk-docs/content/plugins/supported-languages.md deleted file mode 100755 index 46208d93f8..0000000000 --- a/tyk-docs/content/plugins/supported-languages.md +++ /dev/null @@ -1,87 +0,0 @@ ---- -date: 2017-03-24T15:45:13Z -title: Supported Languages -menu: - main: - parent: "Custom Plugins" - ---- - -Tyk recommends using Go plugins for performance, flexibility, and nativity reasons (all Tyk components are written in Go). - -The following languages are supported for custom plugins: -* [Golang native plugins]({{< ref "plugins/supported-languages/golang" >}}) - fast, native performance -* [JavaScript Plugins]({{< ref "plugins/supported-languages/javascript-middleware" >}}) (JSVM Middleware) - simple with limited direct API when performance not important (same with Python / Lua) -* [Rich Plugins]({{< ref "plugins/supported-languages/rich-plugins" >}}) includes Python, Lua, gRPC - With gRPC, you can write plugins in Java, .NET, C++ / C#, PHP, and all other [gRPC supported languages](https://grpc.io/docs/languages/). -Rich plugins give ultimate flexibility in the language of implementation, however, there are some performance and management overheads when compared with native GoLang plugin. - - -**Common To All Plugin Languages:** - -* Make Layer 4 (TCP) or Layer 7 (HTTP/REST/SOAP) calls -* Open Persistent Connections -* Modify the request in-flight -* Used to stop the request and return a [custom response]({{< ref "plugins/plugin-types/request-plugins#return-overrides--returnoverrides" >}}) -* Be served using [Bundles]({{< ref "plugins/how-to-serve-plugins" >}}) or by files on the file system, except gRPC of course which by definition is served by some webserver in the language of your choosing - - -## Plugin Hook Types - -Tyk provide 5 different phases, i.e. hooks to inject custom plugin throughout the [API execution lifecycle]({{< ref "concepts/middleware-execution-order" >}}). There are performance advantages to picking the correct phase, and of course that depends on your use case and what functionality you need. - -Not all hooks are supported in every language. The following table shows you which plugin language support which phase/hook: - -| | Auth | Pre | Post-Auth | Post | Response -|------------|--------|----------|-----------|------|-----------| -| GoLang | ✅ |✅ |✅ |✅ |✅ -| JavaScript | ❌ |✅ |❌ |✅ |❌ -| gRPC | ✅ |✅ |✅ |✅ |✅ -| Python | ✅ |✅ |✅ |✅ |✅ -| Lua | ✅ |✅ |✅ |✅ |❌ - -More reading on the [hook types]({{< ref "plugins/supported-languages/rich-plugins/rich-plugins-work#coprocess-dispatcher---hooks" >}}) in rich plugins and explanation with common use case for each [hook type]({{}}) - - -## Plugin Driver Names -We use the following Plugin driver names: - -| Plugin | Name | -| ---------- | --------- | -| GoLang | goplugin | -| JavaScript | otto | -| gRPC | grpc | -| Python | python | -| Lua | lua | - - -## Limitations - -What are the limitations to using this programming Language? - -| | GoLang | JavaScript | gRPC | Python | Lua -|---|--------|------------------|-----------|-----------|-----------| -| Runs in Gateway process | ✅
Runs
natively | ✅
Built-In JSVM Interpreter | ❌
Standalone server | ✅
Tyk talks with Python interpreter |✅ -| Built-in SDK | ✅
All Gateway Functionality | ✅
[Yes]({{< ref "plugins/supported-languages/javascript-middleware/javascript-api" >}}) | ❌ | ✅
[Yes]({{< ref "plugins/supported-languages/rich-plugins/python/tyk-python-api-methods" >}}) | ❌ -| TCP Connections

(DBs, Redis, etc)

| ✅ | ❌
Very Limited | ✅ | ✅ | ✅ | - -## Custom Plugin Table - -We have put together a [GitHub repo with a table of custom plugins](https://github.com/TykTechnologies/custom-plugins#custom-gateway-plugins) in various languages that you can experiment with. If you would like to submit one that you have developed, feel free to open an issue in the repo. - -## Differences between Rich Plugins and JSVM middleware - -### JavaScript -The JavaScript Virtual Machine provides pluggable middleware that can modify a request on the fly and are designed to augment a running Tyk process, are easy to implement and run inside the Tyk process in a sandboxed *ECMAScript* interpreter. This is good, but there are some drawbacks with the JSVM: - -* **Performance**: JSVM is performant, but is not easy to optimize and is dependent on the [otto interpreter](https://github.com/robertkrimen/otto) - this is not ideal. The JSVM also requires a copy of the interpreter object for each request to be made, which can increase memory footprint. -* **Extensibility**: JSVM is a limited interpreter, although it can use some NPM modules, it isn't NodeJS so writing interoperable code (especially with other DBs) is difficult. -* **TCP Access**: The JSVM has no socket access so working with DB drivers and directly with Redis is not possible. - -### Rich Plugins -Rich Plugins can provide replacements for existing middleware functions (as opposed to augmentation) and are designed to be full-blown, optimized, highly capable services. They enable a full customized architecture to be built that integrates with a user's infrastructure. - -Rich Plugins bring about the following improvements: - -* **Performance**: Run on STDIN (unix pipes), which are extremely fast and run in their own memory space, and so can be optimized for performance way beyond what the JSVM could offer. -* **Extensibility**: By allowing any language to be used so long as GRPC is supported, the extensibility of a CPH is completely open. -* **TCP Access**: Because a plugin is a separate process, it can have it's own low-level TCP connections opens to databases and services. diff --git a/tyk-docs/content/plugins/supported-languages/golang.md b/tyk-docs/content/plugins/supported-languages/golang.md deleted file mode 100644 index 6b5ded19c6..0000000000 --- a/tyk-docs/content/plugins/supported-languages/golang.md +++ /dev/null @@ -1,55 +0,0 @@ ---- -date: 2024-03-04 -title: Golang plugins -menu: - main: - parent: "Supported Languages" -weight: 0 -aliases: - - /plugins/golang-plugins/golang-plugins/ - - /customise-tyk/plugins/golang-plugins/golang-plugins/ - - /plugins/supported-languages/golang/ ---- - -Golang plugins are a very flexible and powerful way to extend the functionality of Tyk by attaching custom logic written in Go to [hooks]({{< ref "plugins/plugin-types/plugintypes" >}}) in the Tyk [middleware chain]({{< ref "concepts/middleware-execution-order" >}}). -The chain of middleware is specific to an API and gets created at API load time. When Tyk Gateway performs an API re-load it also loads any custom middleware and "injects" them into a chain to be called at different stages of the HTTP request life cycle. - -For a quick-start guide to working with Go plugins, start [here]({{< ref "plugins/tutorials/quick-starts/go/quickstart" >}}). - -The [Go plugin writing guide]({{< ref "product-stack/tyk-gateway/advanced-configurations/plugins/golang/writing-go-plugins" >}}) provides details of how to access dynamic data (such as the key session object) from your Go functions. Combining these resources provides you with a powerful set of tools for shaping and structuring inbound traffic to your API. - -## Supported plugin types - -All of Tyk's [custom middleware hooks]({{< ref "plugins/plugin-types/plugintypes" >}}) support Go plugins. They represent different stages in the request and response [middleware chain]({{< ref "concepts/middleware-execution-order" >}}) where custom functionality can be added. - -- **Pre** - supports an array of middlewares to be run before any others (i.e. before authentication) -- **Auth** - this middleware performs custom authentication and adds API key session info into the request context and can be used only if the API definition has both: - - `"use_keyless": false` - - `"use_go_plugin_auth": true` -- **Post-Auth** - supports an array of middleware to be run after authentication; at this point, we have authenticated the session API key for the given key (in the request context) so we can perform any extra checks. This can be used only if the API definition has both: - - `"use_keyless": false` - - an authentication method specified -- **Post** - supports an array of middlewares to be run at the very end of the middleware chain; at this point Tyk is about to request a round-trip to the upstream target -- **Response** - run only at the point the response has returned from a service upstream of the API Gateway; note that the [method signature for Response Go plugins]({{< ref "product-stack/tyk-gateway/advanced-configurations/plugins/golang/writing-go-plugins#creating-a-custom-response-plugin" >}}) is slightly different from the other hook types - -{{< note info >}} -**Note** - -The `use_keyless` and `use_go_plugin_auth` fields are populated automatically with the correct values if you add a plugin to the **Auth** or **Post-Auth** hooks when using the Tyk Dashboard. -{{< /note >}} - -## Upgrading your Tyk Gateway - -When upgrading your Tyk Gateway deployment, you need to re-compile your plugin with the new version. At the moment of loading a plugin, the Gateway will try to find a plugin with the name provided in the API definition. If none is found then it will fall back to search the plugin file with the name: `{plugin-name}_{Gw-version}_{OS}_{arch}.so`. - -Since Tyk v4.1.0, the compiler [automatically]({{< ref "product-stack/tyk-gateway/advanced-configurations/plugins/golang/go-plugin-compiler#output-filename" >}}) creates plugin files following this convention so when you upgrade, say from Tyk v5.2.5 to v5.3.0 you only need to have the plugins compiled for v5.3.0 before performing the upgrade. - -This diagram shows how every Tyk Gateway will search and load the plugin binary that it is compatible with. -{{< img src="/img/plugins/go-plugin-different-tyk-versions.png" alt="APIs Menu" >}} - -## Using custom Go plugins with Tyk Cloud - -The following supporting resources are provided for developing plugins on Tyk Cloud: - -- [Enabling Plugins On The Control Plane](https://tyk.io/docs/tyk-cloud/configuration-options/using-plugins/setup-control-plane/#what-do-i-need-to-do-to-use-plugins) -- [Uploading Your Plugin Bundle To S3 Bucket](https://tyk.io/docs/tyk-cloud#uploading-your-bundle) diff --git a/tyk-docs/content/plugins/supported-languages/javascript-middleware.md b/tyk-docs/content/plugins/supported-languages/javascript-middleware.md deleted file mode 100755 index 5fdfd19391..0000000000 --- a/tyk-docs/content/plugins/supported-languages/javascript-middleware.md +++ /dev/null @@ -1,62 +0,0 @@ ---- -title: JavaScript Middleware -tags: - - JavaScript - - JS - - middleware - - scripting - - JSVM - - plugins - - virtual endpoint - - JSVM - - JavaScript Virtual Machine - - dynamic event handler -description: Using JavaScript with Tyk -date: "2017-03-24T14:45:20Z" -aliases: - - /customise-tyk/plugins/javascript-middleware/ - - /customise-tyk/plugins/javascript-middleware/middleware-execution-order/ - - /plugins/javascript-middleware - - /plugins/supported-languages/javascript-middleware/ - - /plugins/supported-languages/javascript-middleware/ - - /plugins/supported-languages/javascript-middleware/install-middleware/install-middleware - - /plugins/javascript-middleware/install-middleware ---- - -There are three middleware components that can be scripted with Tyk: - -1. **Custom JavaScript plugins**: These execute either *pre* or *post* validation. A *pre* middleware component will execute before any session validation or token validation has taken place, while a *post* middleware component will execute after the request has been passed through all checks and is ready to be proxied upstream. - -2. **Dynamic event handlers**: These components fire on certain API events (see the event handlers section), these are fired Async and do not have a cooldown timer. These are documented [here]({{< ref "/product-stack/tyk-gateway/basic-config-and-security/report-monitor-and-trigger-events/event-webhook-tyk-oas#set-up-a-webhook-event-handler-in-the-tyk-oas-api-definition" >}}). - -3. **Virtual endpoints**: These are powerful programmable middleware invoked towards the end of the request processing chain. Unlike the custom JavaScript plugins, the virtual endpoint terminates the request. These are documented [here]({{< ref "advanced-configuration/compose-apis/virtual-endpoints" >}}). - -The JavaScript (JS) [scripting guide]({{< ref "plugins/supported-languages/javascript-middleware/middleware-scripting-guide" >}}) provides details of how to access dynamic data (such as the key session object) from your JS functions. Combining these resources provides you with a powerful set of tools for shaping and structuring inbound traffic to your API. - -### Declared plugin functions - -JavaScript functions are available globally in the same namespace. So, if you include two or more JSVM plugins that call the same function, the last declared plugin implementation of the function will be returned. - -## Enabling the JavaScript Virtual Machine (JSVM) - -The JavaScript Virtual Machine (JSVM) provided in the Gateway is a traditional ECMAScript5 compatible environment. - -Before you can use JavaScript customization in any component you will need to enable the JSVM. - -You do this by setting `enable_jsvm` to `true` in your `tyk.conf` [file]({{< ref "tyk-oss-gateway/configuration#enable_jsvm" >}}). - -## Installing JavaScript middleware - -Installing middleware is different for different Tyk deployments, for example, in Tyk OSS it is possible to directly specify a path to a file in the API Definition, while in Tyk Self-Managed, we recommend using a directory-based loader. - -We've provided the following guides: - -- [Tyk OSS]({{< ref "plugins/supported-languages/javascript-middleware/install-middleware/tyk-ce" >}}) -- [Tyk Self-Managed]({{< ref "plugins/supported-languages/javascript-middleware/install-middleware/tyk-pro" >}}) -- [Tyk Hybrid]({{< ref "plugins/supported-languages/javascript-middleware/install-middleware/tyk-hybrid" >}}) - -{{< note success >}} -**Note** - -Tyk Cloud Classic does not support custom middleware. -{{< /note >}} diff --git a/tyk-docs/content/plugins/supported-languages/javascript-middleware/install-middleware/tyk-ce.md b/tyk-docs/content/plugins/supported-languages/javascript-middleware/install-middleware/tyk-ce.md deleted file mode 100755 index 2c048a1287..0000000000 --- a/tyk-docs/content/plugins/supported-languages/javascript-middleware/install-middleware/tyk-ce.md +++ /dev/null @@ -1,81 +0,0 @@ ---- -weight: 0 -title: Installing Middleware on Tyk OSS -tags: - - Tyk OSS JS plugin - - jave script plugin - - Javascript Middleware -menu: - main: - parent: Install Middleware -date: "2017-03-24T15:38:11Z" -aliases: - - /plugins/javascript-middleware/install-middleware/tyk-ce ---- - -In order to activate middleware when using Tyk OSS or when using a file-based setup, the middleware needs to be registered as part of your API Definition. Registration of middleware components is relatively simple. - -{{< note success >}} -**Note** - -It is important that your object names are unique. -{{< /note >}} - -{{< note success >}} -**Note** - -This functionality may change in subsequent releases. -{{< /note >}} - -### Enable the JSVM - -Before you can use Javascript Middleware you will need to enable the JSVM - -You can do this by setting `enable_jsvm` to `true` in your `tyk.conf` file. - -Adding the middleware plugin is as simple as adding it to your definition file in the middleware sections: - -```json -... -"event_handlers": {}, -"custom_middleware": { - "pre": [ - { - "name": "sampleMiddleware", - "path": "middleware/sample.js", - "require_session": false - } - ], - "post": [ - { - "name": "sampleMiddleware", - "path": "middleware/sample.js", - "require_session": false - } - ] -}, -"enable_batch_request_support": false, -... -``` - -As you can see, the parameters are all dynamic, so you will need to ensure that the path to your middleware is correct. The configuration sections are as follows: - -- `pre`: Defines a list of custom middleware objects to run *in order* from top to bottom. That will be executed *before* any authentication information is extracted from the header or parameter list of the request. Use middleware in this section to pre-process a request before feeding it through the Tyk middleware. - -- `pre[].name`: The name of the middleware object to call. This is case sensitive, and **must** match the name of the middleware object that was created, so in our example - we created `sampleMiddleware` by calling: - - `var sampleMiddleware = new TykJS.TykMiddleware.NewMiddleware({});` - -- `pre[].path`: The path to the middleware component, this will be loaded into the JSVM when the API is initialised. This means that if you reload an API configuration, the middleware will also be re-loaded. You can hot-swap middleware on reload with no service interruption. - -- `pre[].require_session`: Irrelevant for pre-processor middleware, since no auth data has been extracted by the authentication middleware, it cannot be made available to the middleware. - -- `post`: Defines a list of custom middleware objects to run *in order* from top to bottom. That will be executed *after* the authentication, validation, throttling, and quota-limiting middleware has been executed, just before the request is proxied upstream. Use middleware in this section to post-process a request before sending it to your upstream API. - -- `post[].name`: The name of the middleware object to call. This is case sensitive, and **must** match the name of the middleware object that was created, so in our example - we created `sampleMiddleware` by calling: - - `var sampleMiddleware = new TykJS.TykMiddleware.NewMiddleware({});` - -- `post[].path`: The path to the middleware component, this will be loaded into the JSVM when the API is initialised. This means that if you reload an API configuration, the middleware will also be re-loaded. You can hot-swap middleware on reload with no service interruption. - -- `post[].require_session`: Defaults to `false`, if you require access to the session object, it will be supplied as a `session` variable to your middleware processor function. diff --git a/tyk-docs/content/plugins/supported-languages/javascript-middleware/install-middleware/tyk-hybrid.md b/tyk-docs/content/plugins/supported-languages/javascript-middleware/install-middleware/tyk-hybrid.md deleted file mode 100755 index a38b2efefa..0000000000 --- a/tyk-docs/content/plugins/supported-languages/javascript-middleware/install-middleware/tyk-hybrid.md +++ /dev/null @@ -1,43 +0,0 @@ ---- -weight: 0 -title: Installing Middleware on Tyk Hybrid -menu: - main: - parent: Install Middleware -date: "2017-03-24T15:41:38Z" -aliases: - - /plugins/javascript-middleware/install-middleware/tyk-hybrid ---- - -In some cases middleware references can't be directly embedded in API Definitions (for example, when using the dashboard in a Hybrid install). However, there is an easy way to distribute and enable custom middleware for an API on a Tyk node. - -A second method of loading API Definitions in Tyk nodes is to add them as a directory structure in the Tyk node. Tyk will load the middleware plugins dynamically on host-reload without needing a direct reference to them in the API Definition. - -The directory structure looks as follows: - -```text -middleware - / {API Id} - / pre - / {middlewareObject1Name}.js - / {middlewareObject2Name}.js - / post - / {middlewareObject1Name}_with_session.js - / {middlewareObject2Name}.js -``` - -Tyk will check for a folder that matches the `{API Id}` being loaded, and then load the `pre` and `post` middleware from the respective folders. - -{{< note success >}} -**Note** - -The filename MUST match the object to be loaded exactly. -{{< /note >}} - -If your middleware requires session injection, then append `_with_session` to the filename. - -### Enable the JSVM - -Before you can use Javascript Middleware you will need to enable the JSVM - -You can do this by setting `enable_jsvm` to `true` in your `tyk.conf` file. diff --git a/tyk-docs/content/plugins/supported-languages/javascript-middleware/install-middleware/tyk-pro.md b/tyk-docs/content/plugins/supported-languages/javascript-middleware/install-middleware/tyk-pro.md deleted file mode 100755 index 8130694c14..0000000000 --- a/tyk-docs/content/plugins/supported-languages/javascript-middleware/install-middleware/tyk-pro.md +++ /dev/null @@ -1,43 +0,0 @@ ---- -weight: 0 -title: Installing Middleware on Tyk Self-Managed -menu: - main: - parent: Install Middleware -date: "2017-03-24T15:40:54Z" -aliases: - - /plugins/javascript-middleware/install-middleware/tyk-pro ---- - -In some cases middleware references can't be directly embedded in API Definitions (for example, when using the Tyk Dashboard in an Self-Managed installation). However, there is an easy way to distribute and enable custom middleware for an API in a Tyk node by adding them as a directory structure. - -Tyk will load the middleware plugins dynamically on host-reload without needing a direct reference to them in the API Definition. - -The directory structure should look like this: - -```text -middleware - / {API Id} - / pre - / {middlewareObject1Name}.js - / {middlewareObject2Name}.js - / post - / {middlewareObject1Name}_with_session.js - / {middlewareObject2Name}.js -``` - -Tyk will check for a folder that matches the `API Id` being loaded, and then load the `pre` and `post` middleware from the respective directories. - -{{< note success >}} -**Note** - -The filename MUST match the object to be loaded exactly. -{{< /note >}} - -If your middleware requires session injection, then append `_with_session` to the filename. - -### Enable the JSVM - -Before you can use Javascript Middleware you will need to enable the JSVM. - -You can do this by setting `enable_jsvm` to `true` in your `tyk.conf` file. diff --git a/tyk-docs/content/plugins/supported-languages/javascript-middleware/javascript-api.md b/tyk-docs/content/plugins/supported-languages/javascript-middleware/javascript-api.md deleted file mode 100755 index d3e60bb7d9..0000000000 --- a/tyk-docs/content/plugins/supported-languages/javascript-middleware/javascript-api.md +++ /dev/null @@ -1,92 +0,0 @@ ---- -title: JavaScript API -tags: - - JavaScript - - JS - - middleware - - scripting - - JSVM - - plugins - - javascript API -description: Tyk JavaScript system API -date: "2017-03-24T14:54:24Z" -aliases: - - /customise-tyk/plugins/javascript-middleware/javascript-api/ - - /plugins/javascript-middleware/javascript-api ---- - -This system API provides access to resources outside of the JavaScript Virtual Machine sandbox, the ability to make outbound HTTP requests and access to the key management REST API functions. - -Embedded JavaScript interpreters offer the bare bones of a scripting language, but not necessarily the functions that you would expect, especially with JavaScript, where objects such as `XMLHttpRequest()` are a given. However, those interfaces are actually provided by the browser / DOM that the script engine are executing in. In a similar vein, we have included a series of functions to the JSVM for convenience and give the interpreter more capability. - -This list is regularly revised and any new suggestions should be made in our [Github issue tracker](https://github.com/TykTechnologies/tyk/issues). - -Below is the list of functions currently provided by Tyk. - -- `log(string)`: Calling `log("this message")` will cause Tyk to log the string to Tyk's default logger output in the form `JSVM Log: this message` as an INFO statement. This function is especially useful for debugging your scripts. It is recommended to put a `log()` call at the end of your middleware and event handler module definitions to indicate on load that they have been loaded successfully - see the [example scripts](https://github.com/TykTechnologies/tyk/tree/master/middleware) in your Tyk installation `middleware` directory for more details. -- `rawlog(string)`: Calling `rawlog("this message")` will cause Tyk to log the string to Tyk's default logger output without any additional formatting, like adding prefix or date. This function can be used if you want to have own log format, and parse it later with custom tooling. -- `b64enc` - Encode string to base64 -- `b64dec` - Decode base64 string -- `TykBatchRequest` this function is similar to `TykMakeHttpRequest` but makes use of the Tyk Batch API. See the Batch Requests section of the [Tyk Gateway API]({{< ref "tyk-gateway-api" >}}) for more details. -- `TykMakeHttpRequest(JSON.stringify(requestObject))`: This method is used to make an HTTP request, requests are encoded as JSON for deserialisation in the min binary and translation to a system HTTP call. The request object has the following structure: - -```js -newRequest = { - "Method": "POST", - "Body": JSON.stringify(event), - "Headers": {}, - "Domain": "http://foo.com", - "Resource": "/event/quotas", - "FormData": {"field": "value"} -}; -``` - -{{< note success >}} -**Note** - -If you want to include querystring values, add them as part of the `Domain` property. -{{< /note >}} - -Tyk passes a simplified response back which looks like this: - -```go -type TykJSHttpResponse struct { - Code int - Body string - Headers map[string][]string -} -``` - -The response is JSON string encoded, and so will need to be decoded again before it is usable: - -```js -usableResponse = JSON.parse(response); -log("Response code: " + usableResponse.Code); -log("Response body: " + usableResponse.Body); -``` - -This method does not execute asynchronously, so execution will block until a response is received. - -### Working with the key session object - -To work with the key session object, two functions are provided: `TykGetKeyData` and `TykSetKeyData`: - -- `TykGetKeyData(api_key, api_id)`: Use this method to retrieve a [session object]({{< ref "getting-started/key-concepts/what-is-a-session-object" >}}) for the key and the API provided: - - ```js - // In an event handler, we can get the key idea from the event, and the API ID from the context variable. - var thisSession = JSON.parse(TykGetKeyData(event.EventMetaData.Key, context.APIID)) - log("Expires: " + thisSession.expires) - ``` - -- `TykSetKeyData(api_key, api_id)`: Use this method to write data back into the Tyk session store: - - ```js - // You can modify the object just like with the REST API - thisSession.expires = thisSession.expires + 1000; - - // Use TykSetKeyData to set the key data back in the session store - TykSetKeyData(event.EventMetaData.Key, JSON.stringify(thisSession)); - ``` - -All of these methods are described in functional examples in the Tyk `middleware/` and `event_handlers/` folders. diff --git a/tyk-docs/content/plugins/supported-languages/javascript-middleware/middleware-scripting-guide.md b/tyk-docs/content/plugins/supported-languages/javascript-middleware/middleware-scripting-guide.md deleted file mode 100755 index 436d4e2942..0000000000 --- a/tyk-docs/content/plugins/supported-languages/javascript-middleware/middleware-scripting-guide.md +++ /dev/null @@ -1,327 +0,0 @@ ---- -title: Using JavaScript with Tyk -tags: - - JavaScript - - JS - - middleware - - scripting - - JSVM - - plugins - - virtual endpoint -description: Writing custom JS functions for Tyk middleware -date: "2017-03-24T14:51:42Z" -aliases: - - /tyk-api-gateway-v1-9/javascript-plugins/middleware-scripting/ - - /plugins/javascript-middleware/middleware-scripting-guide ---- - -Tyk's JavaScript Virtual Machine (JSVM) provides a serverless compute function that allows for the execution of custom logic directly within the gateway itself. This can be accessed from [multiple locations]({{< ref "plugins/supported-languages/javascript-middleware" >}}) in the API processing chain and allows significant customization and optimization of your request handling. - -In this guide we will cover the features and resources available to you when creating custom functions, highlighting where there are limitations for the different middleware stages. - -## Scripting basics - -Here we cover various facets that you need to be aware of when creating custom functions for Tyk. - -### Accessing external and dynamic data - -JS functions can be given access to external data objects relating to the API request. These allow for the modification of both the request itself and the session: - -- `request`: an [object]({{< ref "plugins/supported-languages/javascript-middleware/middleware-scripting-guide#the-request-object" >}}) describing the API request that invoked the middleware -- `session`: the key session [object]({{< ref "plugins/supported-languages/javascript-middleware/middleware-scripting-guide#the-session-object" >}}) provided by the client when making the API request -- `config`: an [object]({{< ref "plugins/supported-languages/javascript-middleware/middleware-scripting-guide#the-config-object" >}}) containing fields from the API definition - -{{< note success >}} -**Note** - -There are other ways of accessing and editing a session object using the [Tyk JavaScript API functions]({{< ref "plugins/supported-languages/javascript-middleware/javascript-api#working-with-the-key-session-object" >}}). -{{< /note >}} - -### Creating a middleware component - -Tyk injects a `TykJS` namespace into the JSVM, which can be used to initialise a new middleware component. The JS for each middleware component should be in its own `*.js` file. - -You create a middleware object by calling the `TykJS.TykMiddleware.NewMiddleware({})` constructor with an empty object and then initialising it with your function using the `NewProcessRequest()` closure syntax. This is where you expose the [external data objects]({{< ref "plugins/supported-languages/javascript-middleware/middleware-scripting-guide#accessing-external-and-dynamic-data" >}}) to your custom function. - -{{< note success >}} -**Note** - -- For Custom JS plugins and Dynamic Event Handlers, the source code filename must match the function name -- Virtual Endpoints do not have this limitation -{{< /note >}} - -### Returning from the middleware - -When returning from the middleware, you provide specific return data depending upon the type of middleware. - -#### Returning from Custom JS plugin - -A custom JS plugin can modify fields in the API request and the session metadata, however this is not performed directly within the JSVM so the required updates must be passed out of the JSVM for Tyk to apply the changes. This is a requirement and omitting them can cause the middleware to fail. - -The JS function must provide the `request` and `session.meta_data` objects in the `ReturnData` as follows: - -```js -return sampleMiddleware.ReturnData(request, session.meta_data); -``` - -Custom JS plugins sit in the [middleware processing chain]({{< ref "concepts/middleware-execution-order" >}}) and pass the request onto the next middleware before it is proxied to the upstream. If required, however, a custom JS plugin can terminate the request and provide a custom response to the client if you configure the `ReturnOverrides` in the `request` object, as described [here]({{< ref "plugins/supported-languages/javascript-middleware/middleware-scripting-guide#using-returnoverrides" >}}). - -#### Returning from Virtual Endpoint - -Unlike custom JS plugins, Virtual Endpoints always [terminate the request]({{< ref "advanced-configuration/compose-apis/virtual-endpoints#how-virtual-endpoints-work" >}}) so have a different method of returning from the JS function. - -The function must return a `responseObject`. This is crucial as it determines the HTTP response that will be sent back to the client. The structure of this object is defined to ensure that the virtual endpoint can communicate the necessary response details back to the Tyk Gateway, which then forwards it to the client. - -The `responseObject` has the following structure: - -- `code`: an integer representing the HTTP status code of the response -- `headers`: an object containing key-value pairs representing the HTTP headers of the response -- `body`: a string that represents the body of the response which can be plain text, JSON, or XML, depending on what your API client expects to receive - -You must provide the `responseObject` together with the `session.meta_data` as parameters in a call to `TykJsResponse` as follows: - -```js -return TykJsResponse(responseObject, session.meta_data); -``` - -You can find some examples of how this works [here]({{< ref "advanced-configuration/compose-apis/demo-virtual-endpoint" >}}). - -## JavaScript resources - -JavaScript (JS) functions have access to a [system API]({{< ref "plugins/supported-languages/javascript-middleware/javascript-api" >}}) and [library of functions]({{< ref "plugins/supported-languages/javascript-middleware/middleware-scripting-guide#underscorejs-library" >}}). They can also be given access to certain Tyk data objects relating to the API request. - -The system API provides access to resources outside of the JavaScript Virtual Machine sandbox, the ability to make outbound HTTP requests and access to the key management REST API functions. - -### The `request` object - -The `request` object provides a set of arrays that describe the API request. These can be manipulated and, when changed, will affect the request as it passes through the middleware pipeline. For [virtual endpoints]({{< ref "advanced-configuration/compose-apis/virtual-endpoints" >}}) the request object has a [different structure](#VirtualEndpoint-Request). - -The structure of the `request` object is: - -```typesecript -class ReturnOverrides { - ResponseCode: number = 200; - ResponseBody: string = ""; - ResponseHeaders: string[] = []; -} - -class Request { - Headers: { [key: string]: string[] } = {}; - SetHeaders: { [key: string]: string } = {}; - DeleteHeaders: string[] = []; - Body: string = ""; - URL: string = ""; - AddParams: { [key: string]: string } = {}; - DeleteParams: string[] = []; - ReturnOverrides: ReturnOverrides = new ReturnOverrides(); - IgnoreBody: boolean = false; - Method: string = ""; - RequestURI: string = ""; - Scheme: string = ""; -} -``` - - - -- `Headers`: this is an object of string arrays, and represents the current state of the request header; this object cannot be modified directly, but can be used to read header data -- `SetHeaders`: this is a key-value map that will be set in the header when the middleware returns the object; existing headers will be overwritten and new headers will be added -- `DeleteHeaders`: any header name that is in this list will be deleted from the outgoing request; note that `DeleteHeaders` happens before `SetHeaders` -- `Body`: this represents the body of the request, if you modify this field it will overwrite the request -- `URL`: this represents the path portion of the outbound URL, you can modify this to redirect a URL to a different upstream path -- `AddParams`: you can add parameters to your request here, for example internal data headers that are only relevant to your network setup -- `DeleteParams`: these parameters will be removed from the request as they pass through the middleware; note `DeleteParams` happens before `AddParams` -- `ReturnOverrides`: values stored here are used to stop or halt middleware execution and return an error code -- `IgnoreBody`: if this parameter is set to `true`, the original request body will be used; if set to `false` the `Body` field will be used (`false` is the default behavior) -- `Method`: contains the HTTP method (`GET`, `POST`, etc.) -- `RequestURI`: contains the request URI, including the query string, e.g. `/path?key=value` -- `Scheme`: contains the URL scheme, e.g. `http`, `https` - - -#### Using `ReturnOverrides` - -If you configure values in `request.ReturnOverrides` then Tyk will terminate the request and provide a response to the client when the function completes. The request will not be proxied to the upstream. - -The response will use the parameters configured in `ReturnOverrides`: - -- `ResponseCode` -- `ResponseBody` -- `ResponseHeaders` - -In this example, if the condition is met, Tyk will return `HTTP 403 Access Denied` with the custom header `"X-Error":"the-condition"`: - -```js -var testJSVMData = new TykJS.TykMiddleware.NewMiddleware({}); - -testJSVMData.NewProcessRequest(function(request, session, config) { - // Logic to determine if the request should be overridden - if (someCondition) { - request.ReturnOverrides.response_code = 403; - request.ReturnOverrides.response_body = "Access Denied"; - request.ReturnOverrides.headers = {"X-Error": "the-condition"}; - // This stops the request from proceeding to the upstream - } - return testJSVMData.ReturnData(request, session.meta_data); -}); -``` - -#### The virtual endpoint `request` object {#VirtualEndpoint-Request} - -For [virtual endpoint]({{< ref "advanced-configuration/compose-apis/virtual-endpoints" >}}) functions the structure of a Javascript `request` object is: - -```typescript -class VirtualEndpointRequest { - Body: string = ""; - Headers: { [key: string]: string[] } = {}; - Params: { [key: string]: string[] } = {}; - Scheme: string = ""; - URL: string = ""; -} -``` - -- `Body`: HTTP request body, e.g. `""` -- `Headers`: HTTP request headers, e.g. `"Accept": ["*/*"]` -- `Params`: Decoded query and form parameters, e.g. `{ "confirm": ["true"], "userId": ["123"] }` -- `Scheme`: The scheme of the URL ( e.g. `http` or `https`) -- `URL`: The full URL of the request, e.g `/vendpoint/anything?user_id=123\u0026confirm=true` - -
- -{{< note success >}} -**Note** - -Each query and form parameter within the request is stored as an array field in the `Params` field of the request object. - -Repeated parameter assignments are appended to the corresponding array. For example, a request against `/vendpoint/anything?user_id[]=123&user_id[]=234` would result in a Javascript request object similar to that shown below: - -```javascript -const httpRequest = { - Headers: { - "Accept": ["*/*"], - "User-Agent": ["curl/8.1.2"] - }, - Body: "", - URL: "/vendpoint/anything?user_id[]=123\u0026user_id[]=234", - Params: { - "user_id[]": ["123", "234"] - }, - Scheme: "http" -}; -``` -{{< /note >}} - -### The `session` object - -Tyk uses an internal [session object]({{< ref "getting-started/key-concepts/what-is-a-session-object" >}}) to handle the quota, rate limits, access allowances and auth data of a specific key. JS middleware can be granted access to the session object but there is also the option to disable it as deserialising it into the JSVM is computationally expensive and can add latency. Other than the `meta_data` field, the session object itself cannot be directly edited as it is crucial to the correct functioning of Tyk. - -#### Limitations - -- Custom JS plugins at the [pre-]({{< ref "plugins/plugin-types/request-plugins" >}}) stage do not have access to the session object (as it has not been created yet) -- When scripting for Virtual Endpoints, the `session` data will only be available to the JS function if enabled in the middleware configuration. - -#### Sharing data between middleware using the `session` object - -For different middleware to be able to transfer data between each other, the session object makes available a `meta_data` key/value field that is written back to the session store (and can be retrieved by the middleware down the line) - this data is permanent, and can also be retrieved by the REST API from outside of Tyk using the `/tyk/keys/` method. - -{{< note success >}} -**Note** - -A new JSVM instance is created for *each* API that is managed. Consequently, inter-API communication is not possible via shared methods, since they have different bounds. However, it *is* possible using the session object if a key is shared across APIs. -{{< /note >}} - -### The `config` object - -The third Tyk data object that is made available to the script running in the JSVM contains data from the API Definition. This is read-only and cannot be modified by the JS function. The structure of this object is: - -- `APIID`: the unique identifier for the API -- `OrgID`: the organization identifier -- `config_data`: custom attributes defined in the API description - -#### Adding custom attributes to the API Definition - -When working with Tyk OAS APIs, you can add custom attributes in the `data` object in the `x-tyk-api-gateway.middleware.global.pluginConfig` section of the API definition, for example: - -```json {linenos=true, linenostart=1} -{ - "x-tyk-api-gateway": { - "middleware": { - "global": { - "pluginConfig": { - "data": { - "enabled": true, - "value": { - "foo": "bar" - } - } - } - } - } - } -} -``` - -When working with Tyk Classic APIs, you simply add the attributes in the `config_data` object in the root of the API definition: - -```json {linenos=true, linenostart=1} -{ - "config_data": { - "foo": "bar" - } -} -``` - -### Underscore.js Library - -In addition to our Tyk JavaScript API functions, you also have access to all the functions from the [underscore](http://underscorejs.org) library. - -Underscore.js is a JavaScript library that provides a lot of useful functional programming helpers without extending any built-in objects. Underscore provides over 100 functions that support your favorite functional helpers: - -- map -- filter -- invoke - -There are also more specialized goodies, including: - -- function binding -- JavaScript templating -- creating quick indexes -- deep equality testing - -## Example - -In this basic example, we show the creation and initialisation of a middleware object. Note how the three Tyk data objects (`request`, `session`, `config`) are made available to the function and the two objects that are returned from the function (in case the external objects need to be updated). - -```js {linenos=true, linenostart=1} -/* --- sampleMiddleware.js --- */ - -// Create new middleware object -var sampleMiddleware = new TykJS.TykMiddleware.NewMiddleware({}); - -// Initialise the object with your functionality by passing a closure that accepts -// two objects into the NewProcessRequest() function: -sampleMiddleware.NewProcessRequest(function(request, session, config) { - log("This middleware does nothing, but will print this to your terminal.") - - // You MUST return both the request and session metadata - return sampleMiddleware.ReturnData(request, session.meta_data); -}); -``` diff --git a/tyk-docs/content/plugins/supported-languages/javascript-middleware/waf-js-plugin.md b/tyk-docs/content/plugins/supported-languages/javascript-middleware/waf-js-plugin.md deleted file mode 100755 index 7692e84cd9..0000000000 --- a/tyk-docs/content/plugins/supported-languages/javascript-middleware/waf-js-plugin.md +++ /dev/null @@ -1,149 +0,0 @@ ---- -date: 2017-03-24T15:45:13Z -title: WAF (OSS) ModSecurity Plugin example -menu: - main: - parent: "JavaScript Middleware" -weight: 10 ---- - -### Use Case - -Traditionally, a Web Application Firewall (WAF) would be the first layer requests would hit, before reaching the API gateway. This is not possible if the Gateway has to terminate SSL, for things such as mTLS. - -So what do you do if you still want to run your requests through a WAF to automatically scan for malicious action? We incorporate a WAF as part of the request lifecycle by using Tyk's plugin architecture. - -### Prerequisites - -* Already running Tyk - Community Edition or Pro -* Docker, to run the WAF - -### Disclaimer - -This is NOT a production ready plugin because - -* The JavaScript plugin creates a new connection with the WAF for every request -* The request is not sent over SSL -* The WAF is only sent the query params for inspection. - -For higher performance, the plugin could be written in Golang, and a connection pool would be opened and maintained over SSL - -## Install Steps - -### 1. Turn JSVM on your `tyk.conf` at the root level: - -Turn on JSVM interpreter to allow Tyk to run JavaScript plugins. - -``` -"enable_jsvm": true -``` - -### 2. Place the JavaScript plugin on Tyk file system -Copy the JS Plugin as a local .js file to the Gateway's file system. - -From the Gateway root, this will download the plugin called `waf.js` into the `middleware` directory: -``` -curl https://raw.githubusercontent.com/TykTechnologies/custom-plugins/master/plugins/js-pre-post-waf/waf.js | cat > middleware/waf.js -``` - -(Instructions) -If you are running Tyk in Docker, you can get into Tyk Gateway with `docker exec` -``` -$ docker ps | grep gateway -670039a3e0b8 tykio/tyk-gateway:latest "./entrypoint.sh" 14 minutes ago Up 14 minutes 0.0.0.0:8080->8080/tcp tyk-demo_tyk-gateway_1 - -## copy container name or ID -$ docker exec -it 670039a3e0b8 bash - -## Now SSH'd into Tyk Gateway container and can perform curl -root@670039a3e0b8:/opt/tyk-gateway# ls - -apps entrypoint.sh install middleware templates tyk-gateway.pid tyk.conf.example -coprocess event_handlers js policies tyk tyk.conf utils - -## Download the plugin -root@670039a3e0b8:/opt/tyk-gateway# curl https://raw.githubusercontent.com/TykTechnologies/custom-plugins/master/plugins/js-pre-post-waf/waf.js | cat > middleware/waf.js - - % Total % Received % Xferd Average Speed Time Time Time Current - Dload Upload Total Spent Left Speed -100 1125 100 1125 0 0 3906 0 --:--:-- --:--:-- --:--:-- 3975 - -``` - -[waf.js source](https://raw.githubusercontent.com/TykTechnologies/custom-plugins/master/plugins/js-pre-post-waf/waf.js) - -### 3. Import API definition into Tyk -Copy the following Tyk API definition and import it into your environment. - -[API Definition JSON](https://raw.githubusercontent.com/TykTechnologies/custom-plugins/master/plugins/js-pre-post-waf/apidef.json) - -Here's the important section which adds the plugin to the request lifecycle for this API: -```{.json} -"custom_middleware": { - "pre": [ - { - "name": "Waf", - "path": "./middleware/waf.js" - } - ], -``` - -##### How to Import? -[Tyk Pro](https://tyk.io/docs/tyk-configuration-reference/import-apis/#import-apis-via-the-dashboard) - -[Tyk CE](https://tyk.io/docs/try-out-tyk/tutorials/create-api/) - -### 4. Run WAF ModSecurity Using Docker - -First run ModSecurity with the popular [Core RuleSet](https://coreruleset.org/) in Docker -``` -$ docker run -ti -p 80:80 -e PARANOIA=1 --rm owasp/modsecurity-crs:v3.0 -``` - -Open a second terminal and curl it -``` -$ curl localhost - -hello world -``` - -We should see the request show in the WAF server: - -``` -172.17.0.1 - - [30/Jun/2020:00:56:42 +0000] "GET / HTTP/1.1" 200 12 -``` - -Now try a dirty payload: -``` -$ curl 'localhost/?param=">' - - - -403 Forbidden - -

Forbidden

-

You don't have permission to access / -on this server.
-

- -``` - -Our WAF catches the response and returns a `403`. - - -Now we try through Tyk. - -``` -## Clean requests, should get response from upstream's IP endpoint -$ curl localhost:8080/waf/ip - -{ - "origin": "172.30.0.1, 147.253.129.30" -} - -## WAF will detect malicious payload and instruct Tyk to deny -$ curl 'localhost:8080/waf/ip?param="> -{ - "error": "Bad request!" -} -``` diff --git a/tyk-docs/content/plugins/supported-languages/rich-plugins.md b/tyk-docs/content/plugins/supported-languages/rich-plugins.md deleted file mode 100755 index f6bd44d696..0000000000 --- a/tyk-docs/content/plugins/supported-languages/rich-plugins.md +++ /dev/null @@ -1,30 +0,0 @@ ---- -date: 2017-03-24T13:02:17Z -title: Rich Plugins -menu: - main: - parent: "Supported Languages" -weight: 1 -aliases: - - /plugins/rich-plugins - - /plugins/supported-languages/rich-plugins/ ---- - -Rich plugins make it possible to write powerful middleware for Tyk. Tyk supports: - -* [Python]({{< ref "plugins/supported-languages/rich-plugins/python/python" >}}) -* [gRPC]({{< ref "plugins/supported-languages/rich-plugins/grpc" >}}) -* [Lua]({{< ref "plugins/supported-languages/rich-plugins/luajit" >}}) - -gRPC provides the ability to write plugins using many languages including C++, Java, Ruby and C#. - -The dynamically built Tyk binaries can expose and call Foreign Function Interfaces in guest languages that extend the functionality of a gateway process. - -The plugins are able to directly call some Tyk API functions from within their guest language. They can also be configured so that they hook into various points along the standard middleware chain. - -{{< note success >}} -**Note** - -When using Python plugins, the middleware function names are set globally. So, if you include two or more plugins that implement the same function, the last declared plugin implementation of the function will be returned. We plan to add namespaces in the future. -{{< /note >}} - diff --git a/tyk-docs/content/plugins/supported-languages/rich-plugins/grpc.md b/tyk-docs/content/plugins/supported-languages/rich-plugins/grpc.md deleted file mode 100755 index 65d86c1d60..0000000000 --- a/tyk-docs/content/plugins/supported-languages/rich-plugins/grpc.md +++ /dev/null @@ -1,80 +0,0 @@ ---- -date: 2017-03-24T13:19:52Z -title: gRPC -menu: - main: - parent: "Rich Plugins" -weight: 5 -aliases: - - /customise-tyk/plugins/rich-plugins/grpc/ - - /customise-tyk/plugins/rich-plugins/grpc/ - - /plugins/rich-plugins/grpc - - /plugins/rich-plugins/grpc/grpc-plugins-tyk ---- - -gRPC is a very powerful framework for RPC communication across different [languages](https://www.grpc.io/docs). It was created by Google and makes heavy use of HTTP2 capabilities and the [Protocol Buffers](https://developers.google.com/protocol-buffers/) serialisation mechanism to dispatch and exchange requests between Tyk and your gRPC plugins. - -When it comes to built-in plugins, we have been able to integrate several languages like Python, Javascript & Lua in a native way: this means the middleware you write using any of these languages runs in the same process. At the time of writing, the following languages are supported: C++, Java, Objective-C, Python, Ruby, Go, C# and Node.JS. - -For supporting additional languages we have decided to integrate gRPC connections and perform the middleware operations within a gRPC server that is external to the Tyk process. Please contact us to learn more: - -
- -{{< button_left href="https://tyk.io/contact/" color="green" content="Contact us" >}} - -Tyk has built-in support for gRPC backends, enabling you to build rich plugins using any of the gRPC supported languages. See [gRPC by language](http://www.grpc.io/docs/) for further details. - ---- - -## Architectural overview - -An example architecture is illustrated below. - -{{< img src="/img/diagrams/diagram_docs_gRPC-plugins_why-use-it-for-plugins@2x.png" alt="Using gRPC for plugins" >}} - -Here we can see that Tyk Gateway sends requests to an external Java gRPC server to handle authentication, via a CustomAuth plugin. The flow is as follows: - -- Tyk receives a HTTP request. -- Tyk serialises the request and session into a protobuf message that is dispatched to your gRPC server. -- The gRPC server performs custom middleware operations (for example, any modification of the request object). Each plugin (Pre, PostAuthKey, Post, Response etc.) is handled as separate gRPC request. -- The gRPC server sends the request back to Tyk. -- Tyk proxies the request to your upstream API. - ---- - -## Use cases - -Deploying an external gRPC server to handle plugins provides numerous technical advantages: - -- Allows for independent scalability of the service from the Tyk Gateway. -- Utilizes a custom-designed server tailored to address specific security concerns, effectively mitigating various security risks associated with native plugins. - ---- - -## Limitations - -At the time of writing the following features are currently unsupported and unavailable in the serialised request: -- Client certificiates -- OAuth keys -- For graphQL APIs details concerning the *max_query_depth* is unavailable -- A request query parameter cannot be associated with multiple values - ---- - -## Developer Resources - -The [Protocol Buffers](https://github.com/TykTechnologies/tyk/tree/master/coprocess/proto ) and [bindings](https://github.com/TykTechnologies/tyk/tree/master/coprocess/bindings) provided by Tyk should be used in order for successful ommunication between Tyk Gateway and your gRPC plugin server. Documentation for the protobuf messages is available in the [Rich Plugins Data Structures]({{< ref "plugins/supported-languages/rich-plugins/rich-plugins-data-structures" >}}) page. - -You can generate supporting HTML documentation using the *docs* task in the [Taskfile](https://github.com/TykTechnologies/tyk/blob/master/coprocess/proto/Taskfile.yml) file of the [Tyk repository](https://github.com/TykTechnologies/tyk). This documentation explains the protobuf messages and services that allow gRPC plugins to handle a request made to the Gateway. Please refer to the README file within the proto folder of the tyk repository for further details. - -You may re-use the bindings that were generated for our samples or generate the bindings youself for Go, Python and Ruby, as implemented by the *generate* task in the [Taskfile](https://github.com/TykTechnologies/tyk/blob/master/coprocess/proto/Taskfile.yml) file of the [Tyk repository](https://github.com/TykTechnologies/tyk). - -If you wish to generate bindings for another target language you may generate the bindings yourself. The [Protocol Buffers](https://developers.google.com/protocol-buffers/) and [gRPC documentation](http://www.grpc.io/docs) provide specific requirements and instructions for each language. - ---- - -## What's next? - -See our [getting started]({{< ref "plugins/supported-languages/rich-plugins/grpc/write-grpc-plugin" >}}) guide for an explanation of how to write and configure gRPC plugins. - ---- diff --git a/tyk-docs/content/plugins/supported-languages/rich-plugins/grpc/custom-auth-dot-net.md b/tyk-docs/content/plugins/supported-languages/rich-plugins/grpc/custom-auth-dot-net.md deleted file mode 100644 index 772668d02e..0000000000 --- a/tyk-docs/content/plugins/supported-languages/rich-plugins/grpc/custom-auth-dot-net.md +++ /dev/null @@ -1,318 +0,0 @@ ---- -date: 2023-04-23T16:54:45Z -title: Create Custom Authentication Plugin with .NET -menu: - main: - parent: "gRPC" -weight: 2 -aliases: - - /customise-tyk/plugins/rich-plugins/grpc/custom-auth-dot-net/ - - "/plugins/rich-plugins/grpc/custom-auth-dot-net" ---- - -This tutorial will guide you through the creation of a custom authentication plugin for Tyk with a gRPC based plugin with .NET and C#. For additional information check the official gRPC [documentation](https://grpc.io/docs/guides/index.html). - -The sample code that we’ll use implements a very simple authentication layer using .NET and the proper gRPC bindings generated from our Protocol Buffers definition files. - -{{< img src="/img/diagrams/diagram_docs_gRPC-plugins_why-use-it-for-plugins@2x.png" alt="Using gRPC for plugins" >}} - -## Requirements - -- Tyk Gateway: This can be installed using standard package management tools like Yum or APT, or from source code. See [here][1] for more installation options. -- The Tyk CLI utility, which is bundled with our RPM and DEB packages, and can be installed separately from [https://github.com/TykTechnologies/tyk-cli][2] -- In Tyk 2.8 the Tyk CLI is part of the gateway binary, you can find more information by running "tyk help bundle". -- .NET Core for your OS: https://www.microsoft.com/net/core -- gRPC tools: https://grpc.io/docs/quickstart/csharp.html#generate-grpc-code - -## Create the Plugin - -### Create .NET Project - -We use the .NET CLI tool to generate the initial files for our project: - -```bash -cd ~ -dotnet new console -o tyk-plugin -``` - -We now have a `tyk-plugin` directory containing the basic skeleton of a .NET application. - -From the `tyk-plugin` directory we need to install a few packages that the gRPC server requires: - -```bash -dotnet add package Grpc --version 1.6.0 -dotnet add package System.Threading.ThreadPool --version 4.3.0 -dotnet add package Google.Protobuf --version 3.4.0 -``` - -- The `Grpc` package provides base code for our server implementation. -- The `ThreadPool` package is used by `Grpc`. -- The `Protobuf` package will be used by our gRPC bindings. - -### Install the gRPC Tools - -We need to install the gRPC tools to generate the bindings. We recommended you follow the official guide here: https://grpc.io/docs/quickstart/csharp.html#generate-grpc-code. - -Run the following Commands (both MacOS and Linux): - -```bash -cd ~/tyk-plugin -temp_dir=packages/Grpc.Tools.1.6.x/tmp -curl_url=https://www.nuget.org/api/v2/package/Grpc.Tools/ -mkdir -p $temp_dir && cd $temp_dir && curl -sL $curl_url > tmp.zip; unzip tmp.zip && cd .. && cp -r tmp/tools . && rm -rf tmp && cd ../.. -chmod -Rf +x packages/Grpc.Tools.1.6.x/tools/ -``` - -Then run the following, depending on your OS: - -**MacOS (x64)** - -```bash -export GRPC_TOOLS=packages/Grpc.Tools.1.6.x/tools/macosx_x64 -``` - -**Linux (x64)** - -```bash -export GRPC_TOOLS=packages/Grpc.Tools.1.6.x/tools/linux_x64 -``` - -The `GRPC_TOOLS` environment variable will point to the appropriate GrpcTools path that matches our operating system and architecture. The last step is to export a variable for the `protoc` program; this is the main program used to generate bindings: - -```bash -export GRPC_PROTOC=$GRPC_TOOLS/protoc -``` - -Now that we can safely run `protoc`, we can download the Tyk Protocol Buffers definition files. These files contain the data structures used by Tyk. See [Data Structures]({{< ref "plugins/supported-languages/rich-plugins/rich-plugins-data-structures" >}}) for more information: - -```bash -cd ~/tyk-plugin -git clone https://github.com/TykTechnologies/tyk -``` - -### Generate the bindings - -To generate the bindings, we create an empty directory and run the `protoc` tool using the environment variable that was set before: - -```bash -mkdir Coprocess -$GRPC_PROTOC -I=tyk/coprocess/proto --csharp_out=Coprocess --grpc_out=Coprocess --plugin=protoc-gen-grpc=$GRPC_TOOLS/grpc_csharp_plugin tyk/coprocess/proto/*.proto -``` - -Run the following command to check the binding directory: - -```bash -ls Coprocess -``` - -The output will look like this: - -``` -CoprocessCommon.cs CoprocessObject.cs CoprocessReturnOverrides.cs -CoprocessMiniRequestObject.cs CoprocessObjectGrpc.cs CoprocessSessionState.cs -``` - -### Implement Server - -Create a file called `Server.cs`. - -Add the following code to `Server.cs`. - -```c# -using System; -using System.Threading.Tasks; -using Grpc.Core; - -using Coprocess; - -class DispatcherImpl : Dispatcher.DispatcherBase -{ - public DispatcherImpl() - { - Console.WriteLine("Instantiating DispatcherImpl"); - } - - - // The Dispatch method will be called by Tyk for every configured hook, we'll implement a very simple dispatcher here: - public override Task Dispatch(Coprocess.Object thisObject, ServerCallContext context) - { - // thisObject is the request object: - Console.WriteLine("Receiving object: " + thisObject.ToString()); - - // hook contains the hook name, this will be defined in our plugin bundle and the implementation will be a method in this class (DispatcherImpl), we'll look it up: - var hook = this.GetType().GetMethod(thisObject.HookName); - - // If hook is null then a handler method for this hook isn't implemented, we'll log this anyway: - if (hook == null) - { - Console.WriteLine("Hook name: " + thisObject.HookName + " (not implemented!)"); - // We return the unmodified request object, so that Tyk can proxy this in the normal way. - return Task.FromResult(thisObject); - }; - - // If there's a handler method, let's log it and proceed with our dispatch work: - Console.WriteLine("Hook name: " + thisObject.HookName + " (implemented)"); - - // This will dynamically invoke our hook method, and cast the returned object to the required Protocol Buffers data structure: - var output = hook.Invoke(this, new object[] { thisObject, context }); - return (Task)output; - } - - // MyPreMiddleware implements a PRE hook, it will be called before the request is proxied upstream and before the authentication step: - public Task MyPreMiddleware(Coprocess.Object thisObject, ServerCallContext context) - { - Console.WriteLine("Calling MyPreMiddleware."); - // We'll inject a header in this request: - thisObject.Request.SetHeaders["my-header"] = "my-value"; - return Task.FromResult(thisObject); - } - - // MyAuthCheck implements a custom authentication mechanism, it will initialize a session object if the token matches a certain value: - public Task MyAuthCheck(Coprocess.Object thisObject, ServerCallContext context) - { - // Request.Headers contains all the request headers, we retrieve the authorization token: - var token = thisObject.Request.Headers["Authorization"]; - Console.WriteLine("Calling MyAuthCheck with token = " + token); - - // We initialize a session object if the token matches "abc123": - if (token == "abc123") - { - Console.WriteLine("Successful auth!"); - var session = new Coprocess.SessionState(); - session.Rate = 1000; - session.Per = 10; - session.QuotaMax = 60; - session.QuotaRenews = 1479033599; - session.QuotaRemaining = 0; - session.QuotaRenewalRate = 120; - session.Expires = 1479033599; - - session.LastUpdated = 1478033599.ToString(); - - thisObject.Metadata["token"] = token; - thisObject.Session = session; - return Task.FromResult(thisObject); - - } - - // If the token isn't "abc123", we return the request object in the original state, without a session object, Tyk will reject this request: - Console.WriteLine("Rejecting auth!"); - return Task.FromResult(thisObject); - } -} -``` - -Create a file called `Program.cs` to instantiate our dispatcher implementation and start a gRPC server. - -Add the following code to `Program.cs`. - -```bash -using System; -using Grpc.Core; - -namespace tyk_plugin -{ - class Program - { - - // Port to attach the gRPC server to: - const int Port = 5555; - - static void Main(string[] args) - { - // We initialize a Grpc.Core.Server and attach our dispatcher implementation to it: - Server server = new Server - { - Services = { Coprocess.Dispatcher.BindService(new DispatcherImpl()) }, - Ports = { new ServerPort("localhost", Port, ServerCredentials.Insecure) } - }; - server.Start(); - - Console.WriteLine("gRPC server listening on " + Port); - Console.WriteLine("Press any key to stop the server..."); - Console.ReadKey(); - - server.ShutdownAsync().Wait(); - - } - } -} -``` - -To run the gRPC server use the following command from the plugin directory: - -```bash -dotnet run -``` - -The gRPC server will listen on port 5555 (as defined in `Program.cs`). In the next steps we'll setup the plugin bundle and modify Tyk to connect to our gRPC server. - -## Bundle the Plugin - -We need to create a manifest file within the `tyk-plugin` directory. This file contains information about our plugin and how we expect it to interact with the API that will load it. This file should be named `manifest.json` and needs to contain the following: - -```json -{ - "custom_middleware": { - "driver": "grpc", - "auth_check": { - "name": "MyAuthMiddleware", - "path": "", - "raw_body_only": false, - "require_session": false - } - } -} -``` - -- The `custom_middleware` block contains the middleware settings like the plugin driver we want to use (`driver`) and the hooks that our plugin will expose. We use the `auth_check` hook for this tutorial. For other hooks see [here]({{< ref "plugins/supported-languages/rich-plugins/rich-plugins-work#coprocess-dispatcher---hooks" >}}). -- The `name` field references the name of the function that we implement in our plugin code - `MyAuthMiddleware`. This will be handled by our dispatcher gRPC method (implemented in `Server.cs`). -- The `path` field is the path to the middleware component. -- The `raw_body_only` field -- The `require_session` field, if set to `true` gives you access to the session object. It will be supplied as a session variable to your middleware processor function - - -To bundle our plugin run the following command in the `tyk-plugin` directory. Check your tyk-cli install path first: - -```bash -/opt/tyk-gateway/utils/tyk-cli bundle build -y -``` - -From Tyk v2.8 upwards you can use: -```bash -/opt/tyk-gateway/bin/tyk bundle build -y -``` - -A plugin bundle is a packaged version of the plugin. It may also contain a cryptographic signature of its contents. The `-y` flag tells the Tyk CLI tool to skip the signing process in order to simplify the flow of this tutorial. - -For more information on the Tyk CLI tool, see [here]({{< ref "plugins/how-to-serve-plugins/plugin-bundles" >}}). - -You should now have a `bundle.zip` file in the `tyk-plugin` directory. - -## Publish the Plugin - -To publish the plugin, copy or upload `bundle.zip` to a local web server like Nginx, or Apache or storage like Amazon S3. For this tutorial we'll assume you have a web server listening on `localhost` and accessible through `http://localhost`. - -{{< include "grpc-include" >}} - -## What's Next? - -In this tutorial we learned how Tyk gRPC plugins work. For a production-level setup we suggest the following: - -- Configure an appropriate web server and path to serve your plugin bundles. -- See the following [GitHub repo](https://github.com/TykTechnologies/tyk-plugin-demo-dotnet) for a gRPC based .NET plugin that incorporates authentication based on Microsoft SQL Server. - - - - - - - - - - - -[1]: https://tyk.io/docs/get-started/with-tyk-on-premise/installation/ -[2]: https://github.com/TykTechnologies/tyk-cli -[3]: /img/dashboard/system-management/plugin_options.png -[4]: /img/dashboard/system-management/plugin_auth_mode.png diff --git a/tyk-docs/content/plugins/supported-languages/rich-plugins/grpc/custom-auth-nodejs.md b/tyk-docs/content/plugins/supported-languages/rich-plugins/grpc/custom-auth-nodejs.md deleted file mode 100644 index b88aaf042b..0000000000 --- a/tyk-docs/content/plugins/supported-languages/rich-plugins/grpc/custom-auth-nodejs.md +++ /dev/null @@ -1,221 +0,0 @@ ---- -date: 2017-03-24T13:28:45Z -title: Create Custom Authentication Plugin with NodeJS -menu: - main: - parent: "gRPC" -weight: 3 -aliases: - - "/plugins/rich-plugins/grpc/custom-auth-nodejs" ---- - -This tutorial will guide you through the creation of a custom authentication plugin for Tyk with a gRPC based plugin written in NodeJS. For additional information about gRPC, check the official documentation [here](https://grpc.io/docs/guides/index.html). - -The sample code that we'll use implements a very simple authentication layer using NodeJS and the proper gRPC bindings generated from our Protocol Buffers definition files. - -{{< img src="/img/dashboard/system-management/custom_grpc_authentication.png" alt="gRPC Auth Diagram" >}} - -## Requirements - -- Tyk Gateway: This can be installed using standard package management tools like Yum or APT, or from source code. See [here](https://tyk.io/docs/get-started/with-tyk-on-premise/installation/) for more installation options. -- The Tyk CLI utility, which is bundled with our RPM and DEB packages, and can be installed separately from [https://github.com/TykTechnologies/tyk-cli](https://github.com/TykTechnologies/tyk-cli) -- In Tyk 2.8 and upwards the Tyk CLI is part of the gateway binary, you can find more information by running "tyk help bundle". -- NodeJS v6.x.x [https://nodejs.org/en/download/](https://nodejs.org/en/download/) - -## Create the Plugin - -### Create NodeJS Project - -We will use the NPM tool to initialize our project, follow the steps provided by the `init` command: - -```bash -cd ~ -mkdir tyk-plugin -cd tyk-plugin -npm init -``` - -Now we'll add the gRPC package for this project: - -```bash -npm install --save grpc -``` - -### Install gRPC Tools - -Typically to use gRPC and Protocol Buffers you need to use a code generator and generate bindings for the target language that you're using. For this tutorial we'll skip this step and use the dynamic loader that's provided by the NodeJS gRPC library. This mechanism allows a program to load Protocol Buffers definitions directly from `.proto` files. See [this section](https://grpc.io/docs/tutorials/basic/node.html#loading-service-descriptors-from-proto-files) in the gRPC documentation for more details. - -To fetch the required `.proto` files, you may use an official repository where we keep the Tyk Protocol Buffers definition files: - -```bash -cd ~/tyk-plugin -git clone https://github.com/TykTechnologies/tyk -``` - -### Implement Server - -Now we're ready to implement our gRPC server, create a file called `main.js` in the project's directory - -Add the following code to `main.js`. - -```nodejs -const grpc = require('grpc'), - resolve = require('path').resolve - -const tyk = grpc.load({ - file: 'coprocess_object.proto', - root: resolve(__dirname, 'tyk/coprocess/proto') -}).coprocess - -const listenAddr = '127.0.0.1:5555', - authHeader = 'Authorization' - validToken = '71f6ac3385ce284152a64208521c592b' - -// The dispatch function is called for every hook: -const dispatch = (call, callback) => { - var obj = call.request - // We dispatch the request based on the hook name, we pass obj.request which is the coprocess.Object: - switch (obj.hook_name) { - case 'MyPreMiddleware': - preMiddleware(obj, callback) - break - case 'MyAuthMiddleware': - authMiddleware(obj, callback) - break - default: - callback(null, obj) - break - } -} - -const preMiddleware = (obj, callback) => { - var req = obj.request - - // req is the coprocess.MiniRequestObject, we inject a header using the "set_headers" field: - req.set_headers = { - 'mycustomheader': 'mycustomvalue' - } - - // Use this callback to finish the operation, sending back the modified object: - callback(null, obj) -} - -const authMiddleware = (obj, callback) => { - var req = obj.request - - // We take the value from the "Authorization" header: - var token = req.headers[authHeader] - - // The token should be attached to the object metadata, this is used internally for key management: - obj.metadata = { - token: token - } - - // If the request token doesn't match the "validToken" constant we return the call: - if (token != validToken) { - callback(null, obj) - return - } - - // At this point the token is valid and a session state object is initialized and attached to the coprocess.Object: - var session = new tyk.SessionState() - session.id_extractor_deadline = Date.now() + 100000000000 - obj.session = session - callback(null, obj) -} - -main = function() { - server = new grpc.Server() - server.addService(tyk.Dispatcher.service, { - dispatch: dispatch - }) - server.bind(listenAddr, grpc.ServerCredentials.createInsecure()) - server.start() -} - -main() -``` - - -To run the gRPC server run: - -```bash -node main.js -``` - -The gRPC server will listen on port `5555` (see the `listenAddr` constant). In the next steps we'll setup the plugin bundle and modify Tyk to connect to our gRPC server. - - -## Bundle the Plugin - -We need to create a manifest file within the `tyk-plugin` directory. This file contains information about our plugin and how we expect it to interact with the API that will load it. This file should be named `manifest.json` and needs to contain the following: - -```json -{ - "custom_middleware": { - "driver": "grpc", - "auth_check": { - "name": "MyAuthMiddleware", - "path": "", - "raw_body_only": false, - "require_session": false - } - } -} -``` - -- The `custom_middleware` block contains the middleware settings like the plugin driver we want to use (`driver`) and the hooks that our plugin will expose. We use the `auth_check` hook for this tutorial. For other hooks see [here]({{< ref "plugins/supported-languages/rich-plugins/rich-plugins-work#coprocess-dispatcher---hooks" >}}). -- The `name` field references the name of the function that we implement in our plugin code - `MyAuthMiddleware`. The implemented dispatcher uses a switch statement to handle this hook, and calls the `authMiddleware` function in `main.js`. -- The `path` field is the path to the middleware component. -- The `raw_body_only` field -- The `require_session` field, if set to `true` gives you access to the session object. It will be supplied as a session variable to your middleware processor function - -To bundle our plugin run the following command in the `tyk-plugin` directory. Check your tyk-cli install path first: - -```bash -/opt/tyk-gateway/utils/tyk-cli bundle build -y -``` - -For Tyk 2.8 use: -```bash -/opt/tyk-gateway/bin/tyk bundle build -y -``` - -A plugin bundle is a packaged version of the plugin. It may also contain a cryptographic signature of its contents. The `-y` flag tells the Tyk CLI tool to skip the signing process in order to simplify the flow of this tutorial. - -For more information on the Tyk CLI tool, see [here]({{< ref "plugins/how-to-serve-plugins/plugin-bundles" >}}). - -You should now have a `bundle.zip` file in the `tyk-plugin` directory. - -## Publish the Plugin - -To publish the plugin, copy or upload `bundle.zip` to a local web server like Nginx, Apache or storage like Amazon S3. For this tutorial we'll assume you have a web server listening on `localhost` and accessible through `http://localhost`. - -{{< include "grpc-include" >}} - - -## What's Next? - -In this tutorial we learned how Tyk gRPC plugins work. For a production-level setup we suggest the following: - -- Configure an appropriate web server and path to serve your plugin bundles. - - - - - - - - - - - -[1]: https://tyk.io/docs/get-started/with-tyk-on-premise/installation/ -[2]: https://github.com/TykTechnologies/tyk-cli -[3]: /img/dashboard/system-management/plugin_options.png -[4]: /img/dashboard/system-management/plugin_auth_mode.png - - - - - diff --git a/tyk-docs/content/plugins/supported-languages/rich-plugins/grpc/custom-auth-python.md b/tyk-docs/content/plugins/supported-languages/rich-plugins/grpc/custom-auth-python.md deleted file mode 100644 index 372d469ae9..0000000000 --- a/tyk-docs/content/plugins/supported-languages/rich-plugins/grpc/custom-auth-python.md +++ /dev/null @@ -1,662 +0,0 @@ ---- -date: 2024-05-12T17:00:00Z -title: "Create Custom Authentication Plugin With Python" ---- - -In the realm of API security, HMAC-signed authentication serves as a foundational concept. In this developer-focused blog post, we'll use HMAC-signed authentication as the basis for learning how to write gRPC custom authentication plugins with Tyk Gateway. Why learn how to write Custom Authentication Plugins? - -- **Foundational knowledge**: Writing custom authentication plugins provides foundational knowledge of Tyk's extensibility and customization capabilities. -- **Practical experience**: Gain hands-on experience in implementing custom authentication logic tailored to specific use cases, starting with HMAC-signed authentication. -- **Enhanced control**: Exercise greater control over authentication flows and response handling, empowering developers to implement advanced authentication mechanisms beyond built-in features. - -While Tyk Gateway offers built-in support for HMAC-signed authentication, this tutorial serves as a practical guide for developers looking to extend Tyk's capabilities through custom authentication plugins. It extends the gRPC server that we developed in our [getting started guide]({{< ref "getting-started-python" >}}). - -We will develop a basic gRPC server that implements the Tyk Dispatcher service with a custom authentication plugin to handle authentication keys, signed using the HMAC SHA512 algorithm. Subsequently, you will be able to make a request to your API with a HMAC signed authentication key in the *Authorization* header. Tyk Gateway will intercept the request and forward it to your Python gRPC server for HMAC signature and token verification. - -Our plugin will only verify the key against an expected value. In a production environment it will be necessary to verify the key against Redis storage. - -Before we continue ensure that you have: - -- Read and completed our getting started guide that explains how to implement a basic Python gRPC server to echo the request payload received from Tyk Gateway. This tutorial extends the source code of the tyk_async_server.py file to implement a custom authentication plugin for a HMAC signed authentication key. -- Read our HMAC signatures documentation for an explanation of HMAC signed authentication with Tyk Gateway. A brief summary is given in the HMAC Signed Authentication section below. - - -## HMAC Signed Authentication - -Before diving in further, we will give a brief overview of HMAC signed authentication using our custom authentication plugin. - -- **Client request**: The journey begins with a client requesting access to a protected resource on the Tyk API. -- **HMAC signing**: Before dispatching the request, the client computes an HMAC signature using a secret key and request date, ensuring the payload's integrity. -- **Authorization header**: The HMAC signature, along with essential metadata such as the API key and HMAC algorithm, is embedded within the Authorization header. -- **Tyk Gateway verification**: Upon receipt, Tyk Gateway forwards the request to our gRPC server to execute the custom authentication plugin. This will validate the HMAC signature, ensuring the request's authenticity before proceeding with further processing. - -Requests should be made to an API that uses our custom authentication plugin as follows. A HMAC signed key should be included in the *Authorization* header and a date/time string in the *Date* header. An example request is shown in the curl command below: - -```bash -curl -v -H 'Date: Fri, 03 May 2024 12:00:42 GMT' \ --H 'Authorization: Signature keyId="eyJvcmciOiI1ZTlkOTU0NGExZGNkNjAwMDFkMGVkMjAiLCJpZCI6ImdycGNfaG1hY19rZXkiLCJoIjoibXVybXVyNjQifQ==", \ -algorithm="hmac-sha512",signature="9kwBK%2FyrjbSHJDI7INAhBmhHLTHRDkIe2uRWHEP8bgQFQvfXRksm6t2MHeLUyk9oosWDZyC17AbGeP8EFqrp%2BA%3D%3D"' \ -http://localhost:8080/grpc-custom-auth/get -``` - -From the above example, it should be noted that: - -- The *Date* header contains a date string formatted as follows: *Fri, 03 May 2024 11:06:00 GMT*. -- The *Authorization* header is formatted as *Signature keyId=””, algorithm=””, signature=””* where: - - - **keyId** is a Tyk authentication key. - - **algorithm** is the HMAC algorithm used to sign the signature, *hmac-sha512* or *hmac-sha256*. - - **signature** is the HAMC signature calculated with the date string from the *Date* header, signed with a base64 encoded secret value, using the specified HMAC algorithm. The HMAC signature is then encoded as base64. - -## Prerequisites - -Firstly, we need to create the following: - -- An API configured to use a custom authentication plugin. -- A HMAC enabled key with a configured secret for signing. - -This will enable us to issue a request to test that Tyk Gateway integrates with our custom authentication plugin on the gRPC server. - -#### Create API - -We will create an API served by Tyk Gateway, that will forward requests upstream to https://httpbin.org/. - -The API will have the following parameters configured: - -- **Listen path**: Tyk Gateway will listen to API requests on */grpc-custom-auth/* and will strip the listen path for upstream requests. -- **Target URL**: The target URL will be configured to send requests to *http://httpbin/*. -- **Authentication Mode**: The authentication mode will be configured for custom authentication. This is used to trigger CoProcess (gRPC), Python or JSVM plugins to handle custom authentication. - -You can use the following Tyk Classic API definition to get you started, replacing the *org_id* with the ID of your organization. - -```json -{ - "api_definition": { - "id": "662facb2f03e750001a03500", - "api_id": "6c56dd4d3ad942a94474df6097df67ed", - "org_id": "5e9d9544a1dcd60001d0ed20", - "name": "Python gRPC Custom Auth", - "enable_coprocess_auth": true, - "auth": { - "auth_header_name": "Authorization" - }, - "proxy": { - "preserve_host_header": false, - "listen_path": "/grpc-custom-auth/", - "disable_strip_slash": true, - "strip_listen_path": true, - "target_url": "http://httpbin/" - }, - "version_data": { - "not_versioned": false, - "versions": { - "Default": { - "name": "Default", - "expires": "", - "use_extended_paths": true, - "extended_paths": { - "ignored": [], - "white_list": [], - "black_list": [] - } - } - }, - "default_version": "Default" - }, - "active": true - } -} -``` - -The Tyk API definition above can be imported via Tyk Dashboard. Alternatively, if using Tyk Gateway OSS, a POST request can be made to the *api/apis* endpoint of Tyk Gateway. Consult the [Tyk Gateway Open API Specification documentation]({{< ref "tyk-gateway-api" >}}) for usage. - -An illustrative example using *curl* is given below. Please note that you will need to: - -- Update the location to use the protocol scheme, host and port suitable for your environment. -- Replace the value in the *x-tyk-authorization* header with the secret value in your *tyk.conf* file. -- Replace the *org_id* with the ID of your organization. - -```bash -curl -v \ - --header 'Content-Type: application/json' \ - --header 'x-tyk-authorization: your Gateway admin secret' \ - --location http://localhost:8080/tyk/apis/ \ - --data '{\ - "api_definition": {\ - "id": "662facb2f03e750001a03502",\ - "api_id": "6c56dd4d3ad942a94474df6097df67ef",\ - "org_id": "5e9d9544a1dcd60001d0ed20",\ - "name": "Python gRPC Custom Auth",\ - "enable_coprocess_auth": true,\ - "auth": {\ - "auth_header_name": "Authorization"\ - },\ - "proxy": {\ - "preserve_host_header": false,\ - "listen_path": "/grpc-custom-auth-error/",\ - "disable_strip_slash": true,\ - "strip_listen_path": true,\ - "target_url": "http://httpbin/"\ - },\ - "version_data": {\ - "not_versioned": false,\ - "versions": {\ - "Default": {\ - "name": "Default",\ - "expires": "",\ - "use_extended_paths": true,\ - "extended_paths": {\ - "ignored": [],\ - "white_list": [],\ - "black_list": []\ - }\ - }\ - },\ - "default_version": "Default"\ - },\ - "active": true\ - }\ - }' -``` - -A response similar to that given below will be returned by Tyk Gateway: - -```bash -{ - "key": "f97b748fde734b099001ca15f0346dfe", - "status": "ok", - "action": "added" -} -``` - -#### Create HMAC Key - -We will create an key configured to use HMAC signing, with a secret of *secret*. The key will configured to have access to our test API. - -You can use the following configuration below, replacing the value of the *org_id* with the ID of your organization. - -```bash -{ - "quota_max": 1000, - "quota_renews": 1596929526, - "quota_remaining": 1000, - "quota_reset": 1596843126, - "quota_used": 0, - "org_id": "5e9d9544a1dcd60001d0ed20", - "access_rights": { - "662facb2f03e750001a03500": { - "api_id": "662facb2f03e750001a03500", - "api_name": "Python gRPC Custom Auth", - "versions": ["Default"], - "allowed_urls": [], - "limit": null, - "quota_max": 1000, - "quota_renews": 1596929526, - "quota_remaining": 1000, - "quota_reset": 1596843126, - "quota_used": 0, - "per": 1, - "expires": -1 - } - }, - "enable_detailed_recording": true, - "hmac_enabled": true, - "hmac_string": "secret", - "meta_data": {} -} -``` - -You can use Tyk Gateway’s API to create the key by issuing a POST request to the *tyk/keys* endpoint. Consult the [Tyk Gateway Open API Specification documentation]({{< ref "tyk-gateway-api" >}}) for usage. - -An illustrative example using *curl* is given below. Please note that you will need to: - -- Update the location to use the protocol scheme, host and port suitable for your environment. -- Replace the value in the *x-tyk-authorization* header with the secret value in your *tyk.conf* file. - -Replace the *org_id* with the ID of your organization. - -```bash -curl --location 'http://localhost:8080/tyk/keys/grpc_hmac_key' \ ---header 'x-tyk-authorization: your Gateay admin secret' \ ---header 'Content-Type: application/json' \ ---data '{\ - "alias": "grpc_hmac_key",\ - "quota_max": 1000,\ - "quota_renews": 1596929526,\ - "quota_remaining": 1000,\ - "quota_reset": 1596843126,\ - "quota_used": 0,\ - "org_id": "5e9d9544a1dcd60001d0ed20",\ - "access_rights": {\ - "662facb2f03e750001a03500": {\ - "api_id": "662facb2f03e750001a03500",\ - "api_name": "python-grpc-custom-auth",\ - "versions": ["Default"],\ - "allowed_urls": [],\ - "limit": null,\ - "quota_max": 1000,\ - "quota_renews": 1596929526,\ - "quota_remaining": 1000,\ - "quota_reset": 1596843126,\ - "quota_used": 0,\ - "per": 1,\ - "expires": -1\ - }\ - },\ - "enable_detailed_recording": true,\ - "hmac_enabled": true,\ - "hmac_string": "secret",\ - "meta_data": {}\ -}\ -' -``` - -A response similar to that given below should be returned by Tyk Gateway: - -```json -{ - "key": "eyJvcmciOiI1ZTlkOTU0NGExZGNkNjAwMDFkMGVkMjAiLCJpZCI6ImdycGNfaG1hY19rZXkiLCJoIjoibXVybXVyNjQifQ==", - "status": "ok", - "action": "added", - "key_hash": "a72fcdc09caa86b5" -} -``` - -{{< note success>}} - -**Note** - -Make a note of the key ID given in the response, since we will need this to test our API. -{{< /note >}} - -## Implement Plugin - -Our custom authentication plugin will perform the following tasks: - -- Extract the *Authorization* and *Date* headers from the request object. -- Parse the *Authorization* header to extract the *keyId*, *algorithm* and *signature* attributes. -- Compute the HMAC signature using the specific algorithm and date included in the header. -- Verify that the computed HMAC signature matches the signature included in the *Authorization* header. A 401 error response will be returned if verification fails. Our plugin will only verify the key against an expected value. In a production environment it will be necessary to verify the key against Redis storage. -- Verify that the *keyId* matches an expected value (VALID_TOKEN). A 401 error response will be returned to Tyk Gateway if verification fails. -- If verification of the signature and key passes then update the session with HMAC enabled and set the HMAC secret. Furthermore, add the key to the *Object* metadata. - -Return the request *Object* containing the updated session back to Tyk Gateway. When developing custom authentication plugins it is the responsibility of the developer to update the session state with the token, in addition to setting the appropriate response status code and error message when authentication fails. - -### Import Python Modules - -Ensure that the following Python modules are imported at the top of your *tyk_async_server.py* file: - -```python -import asyncio -import base64 -import hashlib -import hmac -import json -import re -import signal -import logging -import urllib.parse - -import grpc -from google.protobuf.json_format import MessageToJson -from grpc_reflection.v1alpha import reflection -import coprocess_object_pb2_grpc -import coprocess_object_pb2 -from coprocess_common_pb2 import HookType -from coprocess_session_state_pb2 import SessionState -``` - -### Add Constants - -Add the following constants to the top of the *tyk_async_server.py* file, after the import statements: - -```bash -SECRET = "c2VjcmV0" -VALID_TOKEN = "eyJvcmciOiI1ZTlkOTU0NGExZGNkNjAwMDFkMGVkMjAiLCJpZCI6ImdycGNfaG1hY19rZXkiLCJoIjoibXVybXVyNjQifQ==" -``` - -- **SECRET** is a base64 representation of the secret used for HMAC signing. -- **VALID_TOKEN** is the key ID that we will authenticate against. - -The values listed above are designed to align with the examples provided in the *Prerequisites* section, particularly those related to HMAC key generation. If you've made adjustments to the HMAC secret or you've modified the key alias referred to in the endpoint path (for instance, *grpc_hmac_key*), you'll need to update these constants accordingly. - -### Extract headers - -Add the following function to your *tyk_async_server.py* file to extract a dictionary of the key value pairs from the *Authorization* header. We will use a regular expression to extract the key value pairs. - -```python -def parse_auth_header(auth_header: str) -> dict[str,str]: - pattern = r'(\w+)\s*=\s*"([^"]+)"' - - matches = re.findall(pattern, auth_header) - - parsed_data = dict(matches) - - return parsed_data -``` - -### Compute HMAC Signature - -Add the following function to your *tyk_async_server.py* to compute the HMAC signature. - -```python -def generate_hmac_signature(algorithm: str, date_string: str, secret_key: str) -> str: - - if algorithm == "hmac-sha256": - hash_algorithm = hashlib.sha256 - elif algorithm == "hmac-sha512": - hash_algorithm = hashlib.sha512 - else: - raise ValueError("Unsupported hash algorithm") - - base_string = f"date: {date_string}" - - logging.info(f"generating signature from: {base_string}") - hmac_signature = hmac.new(secret_key.encode(), base_string.encode(), hash_algorithm) - - return base64.b64encode(hmac_signature.digest()).decode() -``` - -Our function accepts three parameters: - -- **algorithm** is the HMAC algorithm to use for signing. We will use HMAC SHA256 or HMAC SHA512 in our custom authentication plugin -- **date_string** is the date extracted from the date header in the request sent by Tyk Gateway. -- **secret_key** is the value of the secret used for signing. - -The function computes and returns the HMAC signature for a string formatted as *date: date_string*, where *date_string* corresponds to the value of the *date_string* parameter. The signature is computed using the secret value given in the *secret_key* parameter and the HMAC algorithm given in the *algorithm* parameter. A *ValueError* is raised if the hash algorithm is unrecognized. - -We use the following Python modules in our implementation: - -- hmac Python module to compute the HMAC signature. -- base64 Python module to encode the result. - -### Verify HMAC Signature - -Add the following function to your *tyk_async_server.py* file to verify the HMAC signature provided by the client: - -```python -def verify_hmac_signature(algorithm: str, signature: str, source_string) -> bool: - - expected_signature = generate_hmac_signature(algorithm, source_string, SECRET) - received_signature = urllib.parse.unquote(signature) - - if expected_signature != received_signature: - error = f"Signatures did not match\nreceived: {received_signature}\nexpected: {expected_signature}" - logging.error(error) - else: - logging.info("Signatures matched!") - - return expected_signature == received_signature -``` - -Our function accepts three parameters: - -- **algorithm** is the HMAC algorithm to use for signing. We will use hmac-sha256 or hmac-sha512 in our custom authentication plugin. -- **signature** is the signature string extracted from the *Authorization* header. -- **source_string** is the date extracted from the date header in the request sent by Tyk Gateway. -- **secret_key** is the value of the secret used for signing. - -The function calls *generate_hmac_signature* to verify the signatures match. It returns true if the computed and client HMAC signatures match, otherwise false is returned. - -### Set Error Response - -Add the following helper function to *tyk_async_server.py* to allow us to set the response status and error message if authentication fails. - -```python -def set_response_error(object: coprocess_object_pb2.Object, code: int, message: str) -> None: - object.request.return_overrides.response_code = code - object.request.return_overrides.response_error = message -``` - -Our function accepts the following three parameters: - -- **object** is an instance of the [Object]({{< ref "plugins/supported-languages/rich-plugins/rich-plugins-data-structures#object" >}}) message representing the payload sent by Tyk Gateway to the *Dispatcher* service in our gRPC server. For further details of the payload structure dispatched by Tyk Gateway to a gRPC server please consult our gRPC documentation. -- **code** is the HTTP status code to return in the response. -- **message** is the response message. - -The function modifies the *return_overrides* attribute of the request, updating the response status code and error message. The *return_overrides* attribute is an instance of a [ReturnOverrides]({{< ref "plugins/supported-languages/rich-plugins/rich-plugins-data-structures#returnoverrides" >}}) message that can be used to override the response of a given HTTP request. When this attribute is modified the request is terminated and is not sent upstream. - -### Authenticate - -Add the following to your *tyk_async_server.py* file to implement the main custom authentication function. This parses the headers to extract the signature and date from the request, in addition to verifying the HMAC signature and key: - -```python -def authenticate(object: coprocess_object_pb2.Object) -> coprocess_object_pb2.Object: - keys_to_check = ["keyId", "algorithm", "signature"] - - auth_header = object.request.headers.get("Authorization") - date_header = object.request.headers.get("Date") - - parse_dict = parse_auth_header(auth_header) - - if not all(key in parse_dict for key in keys_to_check) or not all([auth_header, date_header]): - set_response_error(object, 400, "Custom middleware: Bad request") - return object - - try: - signature_valid = verify_hmac_signature( - parse_dict["algorithm"], - parse_dict["signature"], - date_header - ) - except ValueError: - set_response_error(object, 400, "Bad HMAC request, unsupported algorithm") - return object - - if not signature_valid or parse_dict["keyId"] != VALID_TOKEN: - set_response_error(object, 401, "Custom middleware: Not authorized") - else: - new_session = SessionState() - new_session.hmac_enabled = True - new_session.hmac_secret = SECRET - - object.metadata["token"] = VALID_TOKEN - object.session.CopyFrom(new_session) - - return object -``` - -The *Object* payload received from the Gateway is updated and returned as a response from the *Dispatcher* service: - -- If authentication fails then we set the error message and status code for the response accordingly, using our *set_response_error* function. -- If authentication passes then we update the session attribute in the *Object* payload to indicate that HMAC verification was performed and provide the secret used for signing. We also add the verified key to the meta data of the request payload. - -Specifically, our function performs the following tasks: - -- Extracts the *Date* and *Authorization* headers from the request and verifies that the *Authorization* header is structured correctly, using our *parse_auth_header* function. We store the extracted *Authorization* header fields in the *parse_dict* dictionary. If the structure is invalid then a 400 bad request response is returned to Tyk Gateway, using our *set_response_error* function. -- We use our *verify_hmac_signature* function to compute and verify the HMAC signature. A 400 bad request error is returned to the Gateway if HMAC signature verification fails, due to an unrecognized HMAC algorithm. -- A 401 unauthorized error response is returned to the Gateway under the following conditions: - - - The client HMAC signature and the computed HMAC signature do not match. - - The extracted key ID does not match the expected key value in VALID_TOKEN. - -- If HMAC signature verification passed and the key included in the *Authorization* header is valid then we update the *SessionState* instance to indicate that HMAC signature verification is enabled, i.e. *hmac_enabled* is set to true. We also specify the HMAC secret used for signing in the *hmac_secret* field and include the valid token in the metadata dictionary. - -### Integrate Plugin - -Update the *Dispatch* method of the *PythonDispatcher* class in your *tyk_async_server.py* file so that our authenticate function is called when the a request is made by Tyk Gateway to execute a custom authentication (*HookType.CustomKeyCheck*) plugin. - -```python -class PythonDispatcher(coprocess_object_pb2_grpc.DispatcherServicer): - async def Dispatch( - self, object: coprocess_object_pb2.Object, context: grpc.aio.ServicerContext - ) -> coprocess_object_pb2.Object: - - logging.info(f"STATE for {object.hook_name}\n{MessageToJson(object)}\n") - - if object.hook_type == HookType.Pre: - logging.info(f"Pre plugin name: {object.hook_name}") - logging.info(f"Activated Pre Request plugin from API: {object.spec.get('APIID')}") - - elif object.hook_type == HookType.CustomKeyCheck: - logging.info(f"CustomAuth plugin: {object.hook_name}") - logging.info(f"Activated CustomAuth plugin from API: {object.spec.get('APIID')}") - - authenticate(object) - - elif object.hook_type == HookType.PostKeyAuth: - logging.info(f"PostKeyAuth plugin name: {object.hook_name}") - logging.info(f"Activated PostKeyAuth plugin from API: {object.spec.get('APIID')}") - - elif object.hook_type == HookType.Post: - logging.info(f"Post plugin name: {object.hook_name}") - logging.info(f"Activated Post plugin from API: {object.spec.get('APIID')}") - - elif object.hook_type == HookType.Response: - logging.info(f"Response plugin name: {object.hook_name}") - logging.info(f"Activated Response plugin from API: {object.spec.get('APIID')}") - logging.info("--------\n") - - return object -``` - -## Test Plugin - -Create the following bash script, *hmac.sh*, to issue a test request to an API served by Tyk Gateway. The script computes a HMAC signature and constructs the *Authorization* and *Date* headers for a specified API. The *Authorization* header contains the HMAC signature and key for authentication. - -Replace the following constant values with values suitable for your environment: - -- **KEY** represents the key ID for the HMAC signed key that you created at the beginning of this guide. -- **HMAC_SECRET** represents the base64 encoded value of the secret for your HMAC key that you created at the beginning of this guide. -- **BASE_URL** represents the base URL, containing the protocol scheme, host and port number that Tyk Gateway listens to for API requests. -- **ENDPOINT** represents the path of your API that uses HMAC signed authentication. - -```bash -#!/bin/bash - -BASE_URL=http://localhost:8080 -ENDPOINT=/grpc-custom-auth/get -HMAC_ALGORITHM=hmac-sha512 -HMAC_SECRET=c2VjcmV0 -KEY=eyJvcmciOiI1ZTlkOTU0NGExZGNkNjAwMDFkMGVkMjAiLCJpZCI6ImdycGNfaG1hY19rZXkiLCJoIjoibXVybXVyNjQifQ== -REQUEST_URL=${BASE_URL}${ENDPOINT} - - -function urlencode() { - echo -n "$1" | perl -MURI::Escape -ne 'print uri_escape($_)' | sed "s/%20/+/g" -} - -# Set date in expected format -date="$(LC_ALL=C date -u +"%a, %d %b %Y %H:%M:%S GMT")" - -# Generate the signature using hmac algorithm with hmac secret from created Tyk key and -# then base64 encoded -signature=$(echo -n "date: ${date}" | openssl sha512 -binary -hmac "${HMAC_SECRET}" | base64) - -# Ensure the signature is base64 encoded -url_encoded_signature=$(echo -n "${signature}" | perl -MURI::Escape -ne 'print uri_escape($_)' | sed "s/%20/+/g") - -# Output the date, encoded date, signature and the url encoded signature -echo "request: ${REQUEST_URL}" -echo "date: $date" -echo "signature: $signature" -echo "url_encoded_signature: $url_encoded_signature" - -# Make the curl request using headers -printf "\n\n----\n\nMaking request to http://localhost:8080/grpc-custom-auth/get\n\n" -set -x -curl -v -H "Date: ${date}" \ - -H "Authorization: Signature keyId=\"${KEY}\",algorithm=\"${HMAC_ALGORITHM}\",signature=\"${url_encoded_signature}\"" \ - ${REQUEST_URL} -``` - -After creating and saving the script, ensure that it is executable by issuing the following command: - -```bash -chmod +x hmac.sh -``` - -Issue a test request by running the script: - -```bash -./hmac.sh -``` - -Observe the output of your gRPC server. You should see the request payload appear in the console output for the server and your custom authentication plugin should have been triggered. An illustrative example is given below: - -```bash -2024-05-13 12:53:49 INFO:root:STATE for CustomHMACCheck -2024-05-13 12:53:49 { -2024-05-13 12:53:49 "hookType": "CustomKeyCheck", -2024-05-13 12:53:49 "hookName": "CustomHMACCheck", -2024-05-13 12:53:49 "request": { -2024-05-13 12:53:49 "headers": { -2024-05-13 12:53:49 "User-Agent": "curl/8.1.2", -2024-05-13 12:53:49 "Date": "Mon, 13 May 2024 11:53:49 GMT", -2024-05-13 12:53:49 "Host": "localhost:8080", -2024-05-13 12:53:49 "Authorization": "Signature keyId=\"eyJvcmciOiI1ZTlkOTU0NGExZGNkNjAwMDFkMGVkMjAiLCJpZCI6ImdycGNfaG1hY19rZXkiLCJoIjoibXVybXVyNjQifQ==\",algorithm=\"hmac-sha512\",signature=\"e9OiifnTDgi3PW2EGJWfeQXCuhuhi6bGLiGhUTFpjEfgdKmX%2FQOFrePAQ%2FAoSFGU%2FzpP%2FCabmQi4zQDPdRh%2FZg%3D%3D\"", -2024-05-13 12:53:49 "Accept": "*/*" -2024-05-13 12:53:49 }, -2024-05-13 12:53:49 "url": "/grpc-custom-auth/get", -2024-05-13 12:53:49 "returnOverrides": { -2024-05-13 12:53:49 "responseCode": -1 -2024-05-13 12:53:49 }, -2024-05-13 12:53:49 "method": "GET", -2024-05-13 12:53:49 "requestUri": "/grpc-custom-auth/get", -2024-05-13 12:53:49 "scheme": "http" -2024-05-13 12:53:49 }, -2024-05-13 12:53:49 "spec": { -2024-05-13 12:53:49 "bundle_hash": "d41d8cd98f00b204e9800998ecf8427e", -2024-05-13 12:53:49 "OrgID": "5e9d9544a1dcd60001d0ed20", -2024-05-13 12:53:49 "APIID": "6c56dd4d3ad942a94474df6097df67ed" -2024-05-13 12:53:49 } -2024-05-13 12:53:49 } -2024-05-13 12:53:49 -2024-05-13 12:53:49 INFO:root:CustomAuth plugin: CustomHMACCheck -2024-05-13 12:53:49 INFO:root:Activated CustomAuth plugin from API: 6c56dd4d3ad942a94474df6097df67ed -2024-05-13 12:53:49 INFO:root:generating signature from: date: Mon, 13 May 2024 11:53:49 GMT -2024-05-13 12:53:49 INFO:root:Signatures matched! -2024-05-13 12:53:49 INFO:root:-------- -``` - -Try changing the SECRET and/or KEY constants with invalid values and observe the output of your gRPC server. You should notice that authentication fails. An illustrative example is given below: - -``` -2024-05-13 12:56:37 INFO:root:STATE for CustomHMACCheck -2024-05-13 12:56:37 { -2024-05-13 12:56:37 "hookType": "CustomKeyCheck", -2024-05-13 12:56:37 "hookName": "CustomHMACCheck", -2024-05-13 12:56:37 "request": { -2024-05-13 12:56:37 "headers": { -2024-05-13 12:56:37 "User-Agent": "curl/8.1.2", -2024-05-13 12:56:37 "Date": "Mon, 13 May 2024 11:56:37 GMT", -2024-05-13 12:56:37 "Host": "localhost:8080", -2024-05-13 12:56:37 "Authorization": "Signature keyId=\"eyJvcmciOiI1ZTlkOTU0NGExZGNkNjAwMDFkMGVkMjAiLCJpZCI6ImdycGNfaG1hY19rZXkiLCJoIjoibXVybXVyNjQifQ==\",algorithm=\"hmac-sha512\",signature=\"KXhkWOS01nbxuFfK7wEBggkydXlKJswxbukiplboJ2n%2BU6JiYOil%2Bx4OE4edWipg4EcG9T49nvY%2Fc9G0XFJcfg%3D%3D\"", -2024-05-13 12:56:37 "Accept": "*/*" -2024-05-13 12:56:37 }, -2024-05-13 12:56:37 "url": "/grpc-custom-auth/get", -2024-05-13 12:56:37 "returnOverrides": { -2024-05-13 12:56:37 "responseCode": -1 -2024-05-13 12:56:37 }, -2024-05-13 12:56:37 "method": "GET", -2024-05-13 12:56:37 "requestUri": "/grpc-custom-auth/get", -2024-05-13 12:56:37 "scheme": "http" -2024-05-13 12:56:37 }, -2024-05-13 12:56:37 "spec": { -2024-05-13 12:56:37 "bundle_hash": "d41d8cd98f00b204e9800998ecf8427e", -2024-05-13 12:56:37 "OrgID": "5e9d9544a1dcd60001d0ed20", -2024-05-13 12:56:37 "APIID": "6c56dd4d3ad942a94474df6097df67ed" -2024-05-13 12:56:37 } -2024-05-13 12:56:37 } -2024-05-13 12:56:37 -2024-05-13 12:56:37 INFO:root:CustomAuth plugin: CustomHMACCheck -2024-05-13 12:56:37 INFO:root:Activated CustomAuth plugin from API: 6c56dd4d3ad942a94474df6097df67ed -2024-05-13 12:56:37 INFO:root:generating signature from: date: Mon, 13 May 2024 11:56:37 GMT -2024-05-13 12:56:37 ERROR:root:Signatures did not match -2024-05-13 12:56:37 received: KXhkWOS01nbxuFfK7wEBggkydXlKJswxbukiplboJ2n+U6JiYOil+x4OE4edWipg4EcG9T49nvY/c9G0XFJcfg== -2024-05-13 12:56:37 expected: zT17C2tgDCYBJCgFFN/mknf6XydPaV98a5gMPNUHYxZyYwYedIPIhyDRQsMF9GTVFe8khCB1FhfyhpmzrUR2Lw== -``` - -## Summary - -In this guide, we've explained how to write a Python gRPC custom authentication plugin for Tyk Gateway, using HMAC-signed authentication as a practical example. Through clear instructions and code examples, we've provided developers with insights into the process of creating custom authentication logic tailored to their specific API authentication needs. - -While Tyk Gateway already supports HMAC-signed authentication out of the box, this guide goes beyond basic implementation by demonstrating how to extend its capabilities through custom plugins. By focusing on HMAC-signed authentication, developers have gained valuable experience in crafting custom authentication mechanisms that can be adapted and expanded to meet diverse authentication requirements. - -It's important to note that the authentication mechanism implemented in this guide solely verifies the HMAC signature's validity and does not include access control checks against specific API resources. Developers should enhance this implementation by integrating access control logic to ensure authenticated requests have appropriate access permissions. - -By mastering the techniques outlined in this guide, developers are better equipped to address complex authentication challenges and build robust API security architectures using Tyk Gateway's extensibility features. This guide serves as a foundation for further exploration and experimentation with custom authentication plugins, empowering developers to innovate and customize API authentication solutions according to their unique requirements. - ---- - -
\ No newline at end of file diff --git a/tyk-docs/content/plugins/supported-languages/rich-plugins/grpc/getting-started-python.md b/tyk-docs/content/plugins/supported-languages/rich-plugins/grpc/getting-started-python.md deleted file mode 100644 index fe81ba75c0..0000000000 --- a/tyk-docs/content/plugins/supported-languages/rich-plugins/grpc/getting-started-python.md +++ /dev/null @@ -1,559 +0,0 @@ ---- -date: 2024-05-10T11:54:00Z -title: "Getting Started: Creating A Python gRPC Server" ---- - -In the realm of API integration, establishing seamless connections between services is paramount. - -Understanding the fundamentals of gRPC server implementation is crucial, especially when integrating with a Gateway solution like Tyk. This guide aims to provide practical insights into this process, starting with the basic principles of how to implement a Python gRPC server that integrates with Tyk Gateway. - -## Objectives - -By the end of this guide, you will be able to implement a gRPC server that will integrate with Tyk Gateway, setting the stage for further exploration in subsequent parts: - -- Establishing the necessary tools, Python libraries and gRPC service definition for implementing a gRPC server that integrates with Tyk Gateway. -- Developing a basic gRPC server that echoes the request payload to the console, showcasing the core principles of integration. -- Configuring Tyk Gateway to interact with our gRPC server, enabling seamless communication between the two services. - -Before implementing our first gRPC server it is first necessary to understand the service interface that defines how Tyk Gateway integrates with a gRPC server. - - -## Tyk Dispatcher Service - -The *Dispatcher* service, defined in the [coprocess_object.proto](https://github.com/TykTechnologies/tyk/blob/master/coprocess/proto/coprocess_object.proto) file, contains the *Dispatch* RPC method, invoked by Tyk Gateway to request remote execution of gRPC plugins. Tyk Gateway dispatches accompanying data relating to the original client request and session. The service definition is listed below: - -```protobuf -service Dispatcher { - rpc Dispatch (Object) returns (Object) {} - rpc DispatchEvent (Event) returns (EventReply) {} -} -``` - -On the server side, we will implement the *Dispatcher* service methods and a gRPC server to handle requests from Tyk Gateway. The gRPC infrastructure decodes incoming requests, executes service methods and encodes service responses. - -Before we start developing our gRPC server we need to setup our development environment with the supporting libraries and tools. - - -## Prerequisites - -Firstly, we need to download the [Tyk Protocol Buffers](https://github.com/TykTechnologies/tyk/tree/master/coprocess/proto) and install the Python protoc compiler. - -We are going to use the *protoc* compiler to generate the supporting classes and data structures to implement the *Dispatcher* service. - - -### Tyk Protocol Buffers - -Issue the following command to download and extract the Tyk Protocol Buffers from the Tyk GitHub repository: - -```bash -curl -sL "https://github.com/TykTechnologies/tyk/archive/master.tar.gz " -o tyk.tar.gz && \ - mkdir tyk && \ - tar -xzvf tyk.tar.gz --strip-components=1 -C tyk && \ - mv tyk/coprocess/proto/* . && \ - rm -r tyk tyk.tar.gz -``` - -### Install Dependencies - -We are going to setup a Python virtual environment and install some supporting dependencies. Assuming that you have Python [virtualenv](https://virtualenv.pypa.io/en/latest/installation.html) already installed, then issue the following commands to setup a Python virtual environment containing the grpcio and grpcio-tools libraries: - -```bash -python3 -m venv .venv -source .venv/bin/activate -pip install –upgrade pip -pip install grpcio grpcio-tools grpcio-reflection -``` - -The [grpcio](https://pypi.org/project/grpcio/) library offers essential functionality to support core gRPC features such as message serialisation and deserialisation. The [grpcio-tools](https://pypi.org/project/grpcio-tools/) library provides the Python *protoc* compiler that we will use to generate the supporting classes and data structures to implement our gRPC server. The [grpcio-reflection](https://pypi.org/project/grpcio-reflection/) library allows clients to query information about the services and methods provided by a gRPC server at runtime. It enables clients to dynamically discover available services, their RPC methods, in addition to the message types and field names associated with those methods. - - -### Install grpcurl - -Follow the [installation instructions](https://github.com/fullstorydev/grpcurl?tab=readme-ov-file#installation) to install grpcurl. We will use grpcurl to send test requests to our gRPC server. - - -### Generate Python Bindings - -We are now able to generate the Python classes and data structures to allow us to implement our gRPC server. To accomplish this we will use the Python *protoc* command as listed below: - -```bash -python -m grpc_tools.protoc --proto_path=. --python_out=. --grpc_python_out=. *.proto -``` - -This compiles the Protocol Buffer files (*.proto) from the current working directory and generates the Python classes representing the Protocol Buffer messages and services. A series of *.py* files should now exist in the current working directory. We are interested in the *coprocess_object_pb2_grpc.py* file, containing a default implementation of *Tyk’s Dispatcher* service. - -Inspect the generated Python file, *coprocess_object_pb2_grpc.py*, containing the *DispatcherServicer* class: - -```python -class DispatcherServicer(object): - """ GRPC server interface, that must be implemented by the target language """ - def Dispatch(self, request, context): - """ Accepts and returns an Object message """ - context.set_code(grpc.StatusCode.UNIMPLEMENTED) - context.set_details('Method not implemented!') - raise NotImplementedError('Method not implemented!') - def DispatchEvent(self, request, context): - """ Dispatches an event to the target language """ - context.set_code(grpc.StatusCode.UNIMPLEMENTED) - context.set_details('Method not implemented!') - raise NotImplementedError('Method not implemented!') -``` - -This superclass contains a default stub implementation for the **Dispatch** and **DispatchEvent** RPC methods, each defining request and context parameters: - -The *request* parameter allows our server to access the message payload sent by Tyk Gateway. We can use this data, pertaining to the request and session, to process and generate a response. - -The *context* parameter provides additional information and functionalities related to the RPC call, such as timeout limits, cancelation signals etc. This is a [grpc.ServicerContext](https://grpc.github.io/grpc/python/grpc.html#grpc.ServicerContext) or a [grpc.aio.ServicerContext](https://grpc.github.io/grpc/python/grpc_asyncio.html#grpc.aio.ServicerContext), object depending upon whether a synchronous or AsyncIO gRPC server is implemented. - -In the next step we will implement a subclass that will handle requests made by Tyk Gateway for remote execution of custom plugins. - - -## Implement Dispatcher Service - -We will now develop the *Dispatcher* service, adding implementations of the *Dispatch* and *DispatchEvent* methods, to allow our gRPC server to integrate with Tyk Gateway. Before we continue, create a file, *async_server.py*, within the same folder as the generated Protocol Buffer (.proto) files. - - -### Dispatch - -Our implementation of the Dispatch RPC method will deserialize the request payload and output to the console as JSON format. This serves as a useful development and debugging aid, allowing inspection of the request and session state dispatched by Tyk Gateway to our gRPC server. - -Copy and paste the following source code into the *async_server.py* file. Notice that we have used type hinting to aid readability. The type hints are located within the type hint files (.pyi) we generated with the protoc compiler. - - -```python -import asyncio -import grpc -import json -import signal -import logging -from google.protobuf.json_format import MessageToJson -from grpc_reflection.v1alpha import reflection -import coprocess_object_pb2_grpc -import coprocess_object_pb2 -from coprocess_common_pb2 import HookType -from coprocess_session_state_pb2 import SessionState -class PythonDispatcher(coprocess_object_pb2_grpc.DispatcherServicer): - async def Dispatch( - self, object: coprocess_object_pb2.Object, context: grpc.aio.ServicerContext - ) -> coprocess_object_pb2.Object: - logging.info(f"STATE for {object.hook_name}\n{MessageToJson(object)}\n") - if object.hook_type == HookType.Pre: - logging.info(f"Pre plugin name: {object.hook_name}") - logging.info(f"Activated Pre Request plugin from API: {object.spec.get('APIID')}") - elif object.hook_type == HookType.CustomKeyCheck: - logging.info(f"CustomAuth plugin: {object.hook_name}") - logging.info(f"Activated CustomAuth plugin from API: {object.spec.get('APIID')}") - elif object.hook_type == HookType.PostKeyAuth: - logging.info(f"PostKeyAuth plugin name: {object.hook_name}") - logging.info(f"Activated PostKeyAuth plugin from API: {object.spec.get('APIID')}") - elif object.hook_type == HookType.Post: - logging.info(f"Post plugin name: {object.hook_name}") - logging.info(f"Activated Post plugin from API: {object.spec.get('APIID')}") - elif object.hook_type == HookType.Response: - logging.info(f"Response plugin name: {object.hook_name}") - logging.info(f"Activated Response plugin from API: {object.spec.get('APIID')}") - logging.info("--------\n") - return object -``` - -Our *Dispatch* RPC method accepts the two parameters, *object* and *context*. The object parameter allows us to inspect the state and session of the request object dispatched by Tyk Gateway, via accessor methods. The *context* parameter can be used to set timeout limits etc. associated with the RPC call. - -The important takeaways from the source code listing above are: - -- The [MessageToJson](https://googleapis.dev/python/protobuf/latest/google/protobuf/json_format.html#google.protobuf.json_format.MessageToJson) function is used to deserialize the request payload as JSON. -- In the context of custom plugins we access the *hook_type* and *hook_name* attributes of the *Object* message to determine which plugin to execute. -- The ID of the API associated with the request is accessible from the spec dictionary, *object.spec.get('APIID')*. - -An implementation of the *Dispatch* RPC method must return the object payload received from Tyk Gateway. The payload can be modified by the service implementation, for example to add or remove headers and query parameters before the request is sent upstream. - - -### DispatchEvent - -Our implementation of the *DispatchEvent* RPC method will deserialize and output the event payload as JSON. Append the following source code to the *async_server.py* file: - -```python - async def DispatchEvent( - self, event: coprocess_object_pb2.Event, context: grpc.aio.ServicerContext - ) -> coprocess_object_pb2.EventReply: - event = json.loads(event.payload) - http://logging.info (f"RECEIVED EVENT: {event}") - return coprocess_object_pb2.EventReply() -``` - -The *DispatchEvent* RPC method accepts the two parameters, *event* and *context*. The event parameter allows us to inspect the payload of the event dispatched by Tyk Gateway. The context parameter can be used to set timeout limits etc. associated with the RPC call. - -The important takeaways from the source code listing above are: - -- The event data is accessible from the *payload* attribute of the event parameter. -- An implementation of the *DispatchEvent* RPC method must return an instance of *coprocess_object_pb2.EventReply*. - - -## Create gRPC Server - -Finally, we will implement an AsyncIO gRPC server to handle requests from Tyk Gateway to the *Dispatcher* service. We will add functions to start and stop our gRPC server. Finally, we will use *grpcurl* to issue a test payload to our gRPC server to test that it is working. - - -### Develop gRPC Server - -Append the following source code from the listing below to the *async_server.py* file: - -```python -async def serve() -> None: - server = grpc.aio.server() - coprocess_object_pb2_grpc.add_DispatcherServicer_to_server( - PythonDispatcher(), server - ) - listen_addr = "[::]:50051" - SERVICE_NAMES = ( - coprocess_object_pb2.DESCRIPTOR.services_by_name["Dispatcher"].full_name, - reflection.SERVICE_NAME, - ) - - reflection.enable_server_reflection(SERVICE_NAMES, server) - server.add_insecure_port(listen_addr) - - logging.info ("Starting server on %s", listen_addr) - - await server.start() - await server.wait_for_termination() - -async def shutdown_server(server) -> None: - http://logging.info ("Shutting down server...") - await server.stop(None) -``` - -The *serve* function starts the gRPC server, listening for requests on port 50051 with reflection enabled. - -Clients can use reflection to list available services, obtain their RPC methods and retrieve their message types and field names dynamically. This is particularly useful for tooling and debugging purposes, allowing clients to discover server capabilities without prior knowledge of the service definitions. - -{{< note success >}} - -**note** - -A descriptor is a data structure that describes the structure of the messages, services, enums and other elements defined in a .proto file. The purpose of the descriptor is primarily metadata: it provides information about the types and services defined in the protocol buffer definition. The *coprocess_object_pb2.py* file that we generated using *protoc* contains a DESCRIPTOR field that we can use to retrieve this metadata. For further details consult the documentation for the Google's protobuf [FileDescriptor](https://googleapis.dev/python/protobuf/latest/google/protobuf/descriptor.html#google.protobuf.descriptor.FileDescriptor.services_by_name) class. - -{{< /note >}} - -The *shutdown_server* function stops the gRPC server via the *stop* method of the server instance. - -The key takeaways from the source code listing above are: - -- An instance of a gRPC server is created using *grpc.aio.server()*. -- A service implementation should be registered with the gRPC server. We register our *PythonDispatcher* class via *coprocess_object_pb2_grpc.add_DispatcherServicer_to_server(PythonDispatcher(), server)*. -- Reflection can be enabled to allow clients to dynamically discover the services available at a gRPC server. We enabled our *Dispatcher* service to be discovered via *reflection.enable_server_reflection(SERVICE_NAMES, server)*. SERVICE_NAMES is a tuple containing the full names of two gRPC services: the *Dispatcher* service obtained by using the DESCRIPTOR field within the *coprocess_object_pb2* module and the other being the standard reflection service. -- The server instance should be started via invoking and awaiting the *start* and *wait_for_termination* methods of the server instance. -- A port may be configured for the server. In this example we configured an insecure port of 50051 on the server instance via the [add_insecure_port function](https://grpc.github.io/grpc/python/grpc.html#grpc.Server.add_insecure_port). It is also possible to add a secure port via the [add_secure_port](https://grpc.github.io/grpc/python/grpc.html#grpc.Server.add_secure_port) method of the server instance, which accepts the port number in addition to an SSL certificate and key to enable TLS encryption. -- The server instance can be stopped via its stop method. - -Finally, we will allow our server to terminate upon receipt of SIGTERM and SIGINT signals. To achieve this, append the source code listed below to the *async_server.py* file. - -```python -def handle_sigterm(sig, frame) -> None: - asyncio.create_task(shutdown_server(server)) - -async def handle_sigint() -> None: - loop = asyncio.get_running_loop() - for sig in (signal.SIGINT, signal.SIGTERM): - loop.add_signal_handler(sig, loop.stop) - -if __name__ == '__main__': - logging.basicConfig(level=logging.INFO) - server = None - signal.signal(signal.SIGTERM, handle_sigterm) - try: - asyncio.get_event_loop().run_until_complete(serve()) - except KeyboardInterrupt: - pass -``` - - -### Start gRPC Server - -Issue the following command to start the gRPC server: - -```bash -python3 -m async_server -``` - -A message should be output on the console, displaying the port number and confirming that the gRPC server has started. - - -### Test gRPC Server - -To test our gRPC server is working, issue test requests to the *Dispatch* and *DispatchEvent* methods, using *grpcurl*. - - -##### Send Dispatch Request - -Use the *grpcurl* command to send a test dispatch request to our gRPC server: - -```bash -grpcurl -plaintext -d '{ - "hookType": "Pre", - "hookName": "MyPreCustomPluginForBasicAuth", - "request": { - "headers": { - "User-Agent": "curl/8.1.2", - "Host": "tyk-gateway.localhost:8080", - "Authorization": "Basic ZGV2QHR5ay5pbzpwYXN0cnk=", - "Accept": "*/*" - }, - "url": "/basic-authentication-valid/get", - "returnOverrides": { - "responseCode": -1 - }, - "method": "GET", - "requestUri": "/basic-authentication-valid/get", - "scheme": "https" - }, - "spec": { - "bundle_hash": "d41d8cd98f00b204e9800998ecf8427e", - "OrgID": "5e9d9544a1dcd60001d0ed20", - "APIID": "04e911d3012646d97fcdd6c846fafc4b" - } -}' localhost:50051 coprocess.Dispatcher/Dispatch -``` - -Inspect the console output of your gRPC server. It should echo the payload that you sent in the request. - - -##### Send DispatchEvent Request - -Use the grpcurl command to send a test event payload to our gRPC server: - -```bash -grpcurl -plaintext -d '{"payload": "{\"event\": \"test\"}"}' localhost:50051 coprocess.Dispatcher/DispatchEvent -``` - -Inspect the console output of your gRPC server. It should display a log similar to that shown below: - -```bash -INFO:root:RECEIVED EVENT: {'event': 'test'} -``` - -The response received from the server should be an empty event reply, similar to that shown below: - -```bash -grpcurl -plaintext -d '{"payload": "{\"event\": \"test\"}"}' localhost:50051 coprocess.Dispatcher/DispatchEvent -{} -``` - -At this point we have tested, independently of Tyk Gateway, that our gRPC Server can handle an example request payload for gRPC plugin execution. In the next section we will create a test environment for testing that Tyk Gateway integrates with our gRPC server for API requests. - - -## Configure Test Environment - -Now that we have implemented and started a gRPC server, Tyk Gateway needs to be configured to integrate with it. To achieve this we will enable the coprocess feature and configure the URL of the gRPC server. - -We will also create an API so that we can test that Tyk Gateway integrates with our gRPC server. - - -### Configure Tyk Gateway - -Within the root of the *tyk.conf* file, add the following configuration, replacing host and port with values appropriate for your environment: - -```yaml -"coprocess_options": { - "enable_coprocess": true, - "coprocess_grpc_server": "tcp://host:port" -} -``` - -Alternatively, the following environment variables can be set in your .env file: - -```bash -TYK_GW_COPROCESSOPTIONS_ENABLECOPROCESS=true -TYK_GW_COPROCESSOPTIONS_COPROCESSGRPCSERVER=tcp://host:port -``` - -Replace host and port with values appropriate for your environment. - - -### Configure API - -Before testing our gRPC server we will create and configure an API with 2 plugins: - -- **Pre Request**: Named *MyPreRequestPlugin*. -- **Response**: Named *MyResponsePlugin* and configured so that Tyk Gateway dispatches the session state with the request. - -Each plugin will be configured to use the *grpc* plugin driver. - -Tyk Gateway will forward details of an incoming request to the gRPC server, for each of the configured API plugins. - - -##### Tyk Classic API - -gRPC plugins can be configured within the *custom_middleware* section of the Tyk Classic ApiDefinition, as shown in the listing below: - -```yaml -{ - "created_at": "2024-03-231T12:49:52Z", - "api_model": {}, - "api_definition": { - ... - ... - "custom_middleware": { - "pre": [ - { - "disabled": false, - "name": "MyPreRequestPlugin", - "path": "", - "require_session": false, - "raw_body_only": false - } - ], - "post": [], - "post_key_auth": [], - "auth_check": { - "disabled": false, - "name": "", - "path": "", - "require_session": false, - "raw_body_only": false - }, - "response": [ - { - "disabled": false, - "name": "MyResponsePlugin", - "path": "", - "require_session": true, - "raw_body_only": false - } - ], - "driver": "grpc", - "id_extractor": { - "disabled": false, - "extract_from": "", - "extract_with": "", - "extractor_config": {} - } - } -} -``` - -In the above listing, the plugin driver parameter has been configured with a value of *grpc*. Two plugins are configured within the *custom_middleware* section: a *Pre Request* plugin and a *Response* plugin. - -The *Response* plugin is configured with *require_session* enabled, so that Tyk Gateway will send details for the authenticated key / user with the gRPC request. Note, this is not configured for *Pre Request* plugins that are triggered before authentication in the request lifecycle. - - -##### Tyk OAS API - -To quickly get started, a Tyk OAS API schema can be created by importing the infamous [pet store](https://petstore3.swagger.io/api/v3/openapi.json) OAS schema. Then the [findByStatus](https://petstore3.swagger.io/api/v3/pet/findByStatus?status=available) endpoint can be used for testing. - -The resulting Tyk OAS API Definition contains the OAS JSON schema with an *x-tyk-api-gateway* section appended, as listed below. gRPC plugins can be configured within the middleware section of the *x-tyk-api-gateway* that is appended at the end of the OAS schema: - -```yaml -"x-tyk-api-gateway": { - "info": { - "id": "6e2ae9b858734ea37eb772c666517f55", - "dbId": "65f457804773a600011af41d", - "orgId": "5e9d9544a1dcd60001d0ed20", - "name": "Swagger Petstore - OpenAPI 3.0 Custom Authentication", - "state": { - "active": true - } - }, - "upstream": { - "url": "https://petstore3.swagger.io/api/v3/" - }, - "server": { - "listenPath": { - "value": "/custom_auth", - "strip": true - }, - "authentication": { - "enabled": true, - "custom": { - "enabled": true, - "header": { - "enabled": false, - "name": "Authorization" - } - } - } - }, - "middleware": { - "global": { - "pluginConfig": { - "driver": "grpc" - } - }, - "cors": { - "enabled": false, - "maxAge": 24, - "allowedHeaders": [ - "Accept", - "Content-Type", - "Origin", - "X-Requested-With", - "Authorization" - ], - "allowedOrigins": [ - "*" - ], - "allowedMethods": [ - "GET", - "HEAD", - "POST" - ] - }, - "prePlugin": { - "plugins": [ - { - "enabled": true, - "functionName": "MyPreRequestPlugin", - "path": "" - } - ] - }, - "responsePlugin": { - "plugins": [ - { - "enabled": true, - "functionName": "MyResponsePlugin", - "path": "", - "requireSession": true - } - ] - } - } -} -``` - -In the above listing, the plugin driver parameter has been set to *grpc*. Two plugins are configured within the middleware section: a *Pre Request* plugin and a *Response* plugin. - -The *Response* plugin is configured with *requireSession* enabled, so that Tyk Gateway will send details for the authenticated key / user with the gRPC request. Note, this is not configurable for *Pre Request* plugins that are triggered before authentication in the request lifecycle. - -Tyk Gateway will forward details of an incoming request to the gRPC server, for each plugin. - - -## Test API - -We have implemented and configured a gRPC server to integrate with Tyk Gateway. Furthermore, we have created an API that has been configured with two gRPC plugins: a *Pre Request* and *Response* plugin. - -When we issue a request to our API and observe the console output of our gRPC server we should see a JSON representation of the request headers etc. echoed in the terminal. - -Issue a request for your API in the terminal window. For example: - -```bash -curl -L http://.localhost:8080/grpc-http-bin -``` - -Observe the console output of your gRPC server. Tyk Gateway should have dispatched two requests to your gRPC server; a request for the *Pre Request* plugin and a request for the *Response* plugin. - -The gRPC server we implemented echoes a JSON representation of the request payload dispatched by Tyk Gateway. - -Note that this is a useful feature for learning how to develop gRPC plugins and understanding the structure of the request payload dispatched by Tyk Gateway to the gRPC server. However, in production environments care should be taken to avoid inadvertently exposing sensitive data such as secrets in the session. - - -## Summary - -In this guide, we've delved into the integration of a Python gRPC server with Tyk Gateway. - -We have explained how to implement a Python gRPC server and equipped developers with the necessary tools, knowledge and capabilities to effectively utilize Tyk Gateway through gRPC services. - -The following essential groundwork has been covered: - -- Setting up tools, libraries and service definitions for the integration. -- Developing a basic gRPC server with functionality to echo the request payload, received from Tyk Gateway, in JSON format. -- Configuring Tyk Gateway for seamless communication with our gRPC server. diff --git a/tyk-docs/content/plugins/supported-languages/rich-plugins/grpc/performance.md b/tyk-docs/content/plugins/supported-languages/rich-plugins/grpc/performance.md deleted file mode 100755 index 3549b3914e..0000000000 --- a/tyk-docs/content/plugins/supported-languages/rich-plugins/grpc/performance.md +++ /dev/null @@ -1,27 +0,0 @@ ---- -date: 2017-03-24T13:21:13Z -title: gRPC Performance -menu: - main: - parent: "gRPC" -weight: 0 -aliases: - - "/plugins/rich-plugins/grpc/performance" ---- - -These are some benchmarks performed on gRPC plugins. - -gRPC plugins may use different transports, we've tested TCP and Unix Sockets. - -## TCP - -{{< img src="/img/diagrams/tcpResponseTime.png" alt="TCP Response Times" >}} - -{{< img src="/img/diagrams/tcpHitRate.png" alt="TCP Hit Rate" >}} - -## Unix Socket - -{{< img src="/img/diagrams/unixResponseTime.png" alt="Unix Socket Response Times" >}} - - -{{< img src="/img/diagrams/unixHitRate.png" alt="Unix Socket Hit Rate" >}} diff --git a/tyk-docs/content/plugins/supported-languages/rich-plugins/grpc/request-transformation-java.md b/tyk-docs/content/plugins/supported-languages/rich-plugins/grpc/request-transformation-java.md deleted file mode 100644 index 934295a042..0000000000 --- a/tyk-docs/content/plugins/supported-languages/rich-plugins/grpc/request-transformation-java.md +++ /dev/null @@ -1,269 +0,0 @@ ---- -date: 2017-03-24T13:28:45Z -title: Create a Request Transformation Plugin with Java -menu: - main: - parent: "gRPC" -weight: 3 -aliases: - - "/plugins/rich-plugins/grpc/request-transformation-java" ---- - -This tutorial will guide you through the creation of a gRPC-based Java plugin for Tyk. -Our plugin will inject a header into the request before it gets proxied upstream. For additional information about gRPC, check the official documentation [here](https://grpc.io/docs/guides/index.html). - -The sample code that we'll use implements a request transformation plugin using Java and uses the proper gRPC bindings generated from our Protocol Buffers definition files. - -## Requirements - -- Tyk Gateway: This can be installed using standard package management tools like Yum or APT, or from source code. See [here][1] for more installation options. -- The Tyk CLI utility, which is bundled with our RPM and DEB packages, and can be installed separately from [https://github.com/TykTechnologies/tyk-cli][2]. -- In Tyk 2.8 the Tyk CLI is part of the gateway binary, you can find more information by running "tyk help bundle". -- Gradle Build Tool: https://gradle.org/install/. -- gRPC tools: https://grpc.io/docs/quickstart/csharp.html#generate-grpc-code -- Java JDK 7 or higher. - -## Create the Plugin - -### Setting up the Java Project - -We will use the Gradle build tool to generate the initial files for our project: - -```bash -cd ~ -mkdir tyk-plugin -cd tyk-plugin -gradle init -``` - -We now have a `tyk-plugin` directory containing the basic skeleton of our application. - -Add the following to `build.gradle` - -```{.copyWrapper} -buildscript { - repositories { - jcenter() - } - dependencies { - classpath 'com.google.protobuf:protobuf-gradle-plugin:0.8.1' - } -} - -plugins { - id "com.google.protobuf" version "0.8.1" - id "java" - id "application" - id "idea" -} - -protobuf { - protoc { - artifact = "com.google.protobuf:protoc:3.3.0" - } - plugins { - grpc { - artifact = 'io.grpc:protoc-gen-grpc-java:1.5.0' - } - } - generateProtoTasks { - all()*.plugins { - grpc {} - } - } - generatedFilesBaseDir = "$projectDir/src/generated" -} - -sourceCompatibility = 1.8 -targetCompatibility = 1.8 - -mainClassName = "com.testorg.testplugin.PluginServer" - -repositories { - mavenCentral() -} - -dependencies { - compile 'io.grpc:grpc-all:1.5.0' -} - -idea { - module { - sourceDirs += file("${projectDir}/src/generated/main/java"); - sourceDirs += file("${projectDir}/src/generated/main/grpc"); - } -} -``` - -### Create the Directory for the Server Class - -```bash -cd ~/tyk-plugin -mkdir -p src/main/java/com/testorg/testplugin -``` - -### Install the gRPC Tools - -We need to download the Tyk Protocol Buffers definition files, these files contains the data structures used by Tyk. See [Data Structures]({{< ref "plugins/supported-languages/rich-plugins/rich-plugins-data-structures" >}}) for more information: - -```bash -cd ~/tyk-plugin -git clone https://github.com/TykTechnologies/tyk -mv tyk/coprocess/proto src/main/proto -``` - -### Generate the Bindings - -To generate the Protocol Buffers bindings we use the Gradle build task: - -```bash -gradle build -``` - -If you need to customize any setting related to the bindings generation step, check the `build.gradle` file. - -### Implement Server - -We need to implement two classes: one class will contain the request dispatcher logic and the actual middleware implementation. The other one will implement the gRPC server using our own dispatcher. - -From the `~/tyk-plugin/src/main/java/com/testorg/testplugin` directory, create a file named `PluginDispatcher.java` with the following code: - -```java -package com.testorg.testplugin; - -import coprocess.DispatcherGrpc; -import coprocess.CoprocessObject; - -public class PluginDispatcher extends DispatcherGrpc.DispatcherImplBase { - - @Override - public void dispatch(CoprocessObject.Object request, - io.grpc.stub.StreamObserver responseObserver) { - CoprocessObject.Object modifiedRequest = null; - - switch (request.getHookName()) { - case "MyPreMiddleware": - modifiedRequest = MyPreHook(request); - default: - // Do nothing, the hook name isn't implemented! - } - - // Return the modified request (if the transformation was done): - if (modifiedRequest != null) { - responseObserver.onNext(modifiedRequest); - }; - - responseObserver.onCompleted(); - } - - CoprocessObject.Object MyPreHook(CoprocessObject.Object request) { - CoprocessObject.Object.Builder builder = request.toBuilder(); - builder.getRequestBuilder().putSetHeaders("customheader", "customvalue"); - return builder.build(); - } -} -``` - -In the same directory, create a file named `PluginServer.java` with the following code. This is the server implementation: - -```java -package com.testorg.testplugin; - -import coprocess.DispatcherGrpc; - -import io.grpc.Server; -import io.grpc.ServerBuilder; -import io.grpc.stub.StreamObserver; -import java.io.IOException; -import java.util.logging.Level; -import java.util.logging.Logger; - -public class PluginServer { - - private static final Logger logger = Logger.getLogger(PluginServer.class.getName()); - static Server server; - static int port = 5555; - - public static void main(String[] args) throws IOException, InterruptedException { - System.out.println("Initializing gRPC server."); - - // Our dispatcher is instantiated and attached to the server: - server = ServerBuilder.forPort(port) - .addService(new PluginDispatcher()) - .build() - .start(); - - blockUntilShutdown(); - - } - - static void blockUntilShutdown() throws InterruptedException { - if (server != null) { - server.awaitTermination(); - } - } -} -``` - -To run the gRPC server we can use the following command: - -```bash -cd ~/tyk-plugin -gradle runServer -``` - -The gRPC server will listen on port 5555 (as defined in `Server.java`). In the next steps we'll setup the plugin bundle and modify Tyk to connect to our gRPC server. - - -## Bundle the Plugin - -We need to create a manifest file within the `tyk-plugin` directory. This file contains information about our plugin and how we expect it to interact with the API that will load it. This file should be named `manifest.json` and needs to contain the following: - -```json -{ - "custom_middleware": { - "driver": "grpc", - "pre": [{ - "name": "MyPreMiddleware" - }] - } -} -``` - -- The `custom_middleware` block contains the middleware settings like the plugin driver we want to use (`driver`) and the hooks that our plugin will expose. We use the `pre` hook for this tutorial. For other hooks see [here]({{< ref "plugins/supported-languages/rich-plugins/rich-plugins-work#coprocess-dispatcher---hooks" >}}). -- The `name` field references the name of the function that we implemented in our plugin code - `MyPreMiddleware`. This will be handled by our dispatcher gRPC method in `PluginServer.java`. - -To bundle our plugin run the following command in the `tyk-plugin` directory. Check your tyk-cli install path first: - -```bash -/opt/tyk-gateway/utils/tyk-cli bundle build -y -``` - -For Tyk 2.8 use: -```bash -/opt/tyk-gateway/bin/tyk bundle build -y -``` - -A plugin bundle is a packaged version of the plugin. It may also contain a cryptographic signature of its contents. The `-y` flag tells the Tyk CLI tool to skip the signing process in order to simplify the flow of this tutorial. - -For more information on the Tyk CLI tool, see [here]({{< ref "plugins/how-to-serve-plugins/plugin-bundles" >}}). - -You should now have a `bundle.zip` file in the `tyk-plugin` directory. - -## Publish the Plugin - -To publish the plugin, copy or upload `bundle.zip` to a local web server like Nginx, or Apache or storage like Amazon S3. For this tutorial we'll assume you have a web server listening on `localhost` and accessible through `http://localhost`. - -{{< include "grpc-include" >}} - - -## What's Next? - -In this tutorial we learned how Tyk gRPC plugins work. For a production-level setup we suggest the following: - -- Configure an appropriate web server and path to serve your plugin bundles. - -[1]: https://tyk.io/docs/get-started/with-tyk-on-premise/installation/ -[2]: https://github.com/TykTechnologies/tyk-cli -[3]: /img/dashboard/system-management/api_settings.png -[4]: /img/dashboard/system-management/plugin_options.png diff --git a/tyk-docs/content/plugins/supported-languages/rich-plugins/grpc/write-grpc-plugin.md b/tyk-docs/content/plugins/supported-languages/rich-plugins/grpc/write-grpc-plugin.md deleted file mode 100755 index c4ccaaa1f8..0000000000 --- a/tyk-docs/content/plugins/supported-languages/rich-plugins/grpc/write-grpc-plugin.md +++ /dev/null @@ -1,321 +0,0 @@ ---- -date: 2017-03-24T13:32:12Z -title: Key Concepts -menu: - main: - parent: "gRPC" -weight: 1 -aliases: - - /plugins/supported-languages/rich-plugins/grpc/write-grpc-plugin - - /plugins/rich-plugins/grpc/write-grpc-plugin - - /plugins/supported-languages/rich-plugins/grpc/tutorial-add-grpc-plugin-api - - /plugins/rich-plugins/grpc/tutorial-add-grpc-plugin-api ---- - -This document serves as a developer's guide for understanding the key concepts and practical steps for writing and configuring gRPC plugins for Tyk Gateway. It provides technical insights and practical guidance to seamlessly integrate Tyk plugins into your infrastructure through gRPC. The goal is to equip developers with the knowledge and tools needed to effectively utilize gRPC for enhancing Tyk Gateway functionalities. - -This comprehensive guide covers essential tasks, including: - -1. **Developing a gRPC Server:** Learn how to develop a gRPC server using [Tyk protocol buffers](https://github.com/TykTechnologies/tyk/tree/master/coprocess/proto). The gRPC server facilitates the execution of Tyk plugins, which offer custom middleware for various phases of the API request lifecycle. By integrating these plugins, developers can enable Tyk Gateway with enhanced control and flexibility in managing API requests, allowing for fine-grained customization and tailored processing at each stage of the request lifecycle. - -2. **Configuring Tyk Gateway:** Set up Tyk Gateway to communicate with your gRPC Server and, optionally, an external secured web server hosting the gRPC plugin bundle for API configurations. Configure Tyk Gateway to fetch the bundle configured for an API from the web server, enabling seamless integration with gRPC plugins. Specify connection settings for streamlined integration. - -3. **API Configuration:** Customize API settings within Tyk Gateway to configure gRPC plugin utilization. Define plugin hooks directly within the API Definition or remotely via an external web server for seamless request orchestration. Tyk plugins provide custom middleware for different phases of the API request lifecycle, enhancing control and flexibility. - -4. **API Testing:** Test that Tyk Gateway integrates with your gRPC server for the plugins configured for your API. - ---- - -## Develop gRPC server - -Develop your gRPC server, using your preferred language, to handle requests from Tyk Gateway for each of the required plugin hooks. These hooks allow Tyk Gateway to communicate with your gRPC server to execute custom middleware at various stages of the API request lifecycle. - -### Prerequisites - -The following prerequisites are necessary for developing a gRPC server that integrates with Tyk Gateway. - -##### Tyk gRPC Protocol Buffers - -A collection of [Protocol Buffer](https://github.com/TykTechnologies/tyk/tree/master/coprocess/proto) messages are available in the Tyk Gateway repository to allow Tyk Gateway to integrate with your gRPC server, requesting execution of plugin code. These messages establish a standard set of data structures that are serialised between Tyk Gateway and your gRPC Server. Developers should consult the [Rich Plugins Data Structures]({{< ref "plugins/supported-languages/rich-plugins/rich-plugins-data-structures" >}}) page for further details. - -##### Protocol Buffer Compiler - -The protocol buffer compiler, `protoc`, should be installed to generate the service and data structures in your preferred language(s) from the [Tyk gRPC Protocol Buffer](https://github.com/TykTechnologies/tyk/tree/master/coprocess/proto) files. Developers should consult the [installation](https://grpc.io/docs/protoc-installation/) documentation at [grpc.io](https://grpc.io/) for an explanation of how to install `protoc`. - -### Generate Bindings - -Generate the bindings (service and data structures) for your target language using the `protoc` compiler. Tutorials are available at [protobuf.dev](https://protobuf.dev/getting-started/) for your target language. - -### Implement service - -Your gRPC server should implement the *Dispatcher* service to enable Tyk Gateway to integrate with your gRPC server. The Protocol Buffer definition for the *Dispatcher* service is listed below: - -```protobuf -service Dispatcher { - rpc Dispatch (Object) returns (Object) {} - rpc DispatchEvent (Event) returns (EventReply) {} -} -``` - -The *Dispatcher* service contains two RPC methods, *Dispatch* and *DispatchEvent*. Dispatch handles a requests made by Tyk Gateway for each plugin configured in your API. DispatchEvent receives notification of an event. - -Your *Dispatch* RPC should handle the request made by Tyk Gateway, implementing custom middleware for the intended plugin hooks. Each plugin hook allows Tyk Gateway to communicate with your gRPC server to execute custom middleware at various stages of the API request lifecycle, such as Pre, PostAuth, Post, Response etc. The Tyk Protocol Buffers define the [HookType](https://github.com/TykTechnologies/tyk/blob/master/coprocess/proto/coprocess_common.proto) enumeration to inspect the type of the intended gRPC plugin associated with the request. This is accessible as an attribute on the *Object* message, e.g. *object_message_instance.hook_type*. - -### Developer resources - -Consult the [Tyk protocol buffers](https://github.com/TykTechnologies/tyk/tree/master/coprocess/proto) for the definition of the service and data structures that enable integration of Tyk gateway with your gRPC server. Tyk provides pre-generated [bindings](https://github.com/TykTechnologies/tyk/tree/master/coprocess/bindings) for C++, Java, Python and Ruby. - -Example tutorials are available that explain how to generate the protobuf bindings and implement a server for [Java]({{< ref "plugins/supported-languages/rich-plugins/grpc/request-transformation-java" >}}), [.NET]({{< ref "plugins/supported-languages/rich-plugins/grpc/custom-auth-dot-net" >}}) and [NodeJS]({{< ref "plugins/supported-languages/rich-plugins/grpc/custom-auth-nodejs" >}}). - -Tyk Github repositories are also available with examples for [Ruby](https://github.com/TykTechnologies/tyk-plugin-demo-ruby) and [C#/.NET](https://github.com/TykTechnologies/tyk-plugin-demo-dotnet) - ---- - -## Configure Tyk Gateway - -Configure Tyk Gateway to issue requests to your gRPC server and optionally, specify the URL of the web server that will serve plugin bundles. - -### Configure gRPC server - -Modify the root of your `tyk.conf` file to include the *coprocess_options* section, similar to that listed below: - -```yaml -"coprocess_options": { - "enable_coprocess": true, - "coprocess_grpc_server": "tcp://127.0.0.1:5555", - "grpc_authority": "localhost", - "grpc_recv_max_size": 100000000, - "grpc_send_max_size": 100000000 -}, -``` - -A gRPC server can configured under the `coprocess_options` section as follows: - -- `enable_coprocess`: Enables the rich plugins feature. -- `coprocess_grpc_server`: Specifies the gRPC server URL, in this example we're using TCP. Tyk will attempt a connection on startup and keep reconnecting in case of failure. -- `grpc_recv_max_size`: Specifies the message size supported by the gateway gRPC client, for receiving gRPC responses. -- `grpc_send_max_size`: Specifies the message size supported by the gateway gRPC client for sending gRPC requests. -- `grpc_authority`: The `authority` header value, defaults to `localhost` if omitted. Allows configuration according to [RFC 7540](https://datatracker.ietf.org/doc/html/rfc7540#section-8.1.2.3). - -When using gRPC plugins, Tyk acts as a gRPC client and dispatches requests to your gRPC server. gRPC libraries usually set a default maximum size, for example, the official gRPC Java library establishes a 4 -MB message size [https://jbrandhorst.com/post/grpc-binary-blob-stream/](https://jbrandhorst.com/post/grpc-binary-blob-stream/). - -Configuration parameters are available for establishing a message size in both directions (send and receive). For most use cases and especially if you're dealing with multiple hooks, where the same request object is dispatched, it is recommended to set both values to the same size. - -### Configure Web server (optional) - -Tyk Gateway can be configured to download the gRPC plugin configuration for an API from a web server. For further details related to the concept of bundling plugins please refer to [plugin bundles]({{< ref "plugins/how-to-serve-plugins/plugin-bundles" >}}). - -```yaml -"enable_bundle_downloader": true, -"bundle_base_url": "https://my-bundle-server.com/bundles/", -"public_key_path": "/path/to/my/pubkey", -``` - -The following parameters can be configured: -- `enable_bundle_downloader`: Enables the bundle downloader to download bundles from a webserver. -- `bundle_base_url`: Base URL from which to serve bundled plugins. -- `public_key_path`: Public key for bundle verification (optional) - -The `public_key_path` value is used for verifying signed bundles, you may omit this if unsigned bundles are used. - ---- - -## Configure API - -Plugin hooks for your APIs in Tyk can be configured either by directly specifying them in a configuration file on the Gateway server or by hosting the configuration externally on a web server. This section explains how to configure gRPC plugins for your API endpoints on the local Gateway or remotely from an external secured web server. - -### Local - -This section provides examples for how to configure gRPC plugin hooks, locally within an API Definition. Examples are provided for Tyk Gateway and Tyk Operator. - -#### Tyk Gateway - -For configurations directly embedded within the Tyk Gateway, plugin hooks can be defined within your API Definition. An example snippet from a Tyk Classic API Definition is provided below: - -```yaml -"custom_middleware": { - "pre": [ - {"name": "MyPreMiddleware"} - ], - "post": [ - {"name": "MyPostMiddleware"} - ], - "auth_check": { - "name": "MyAuthCheck" - }, - "driver": "grpc" -} -``` - -For example, a Post request plugin hook has been configured with name `MyPostMiddleware`. Before the request is sent upstream Tyk Gateway will serialize the request into a [Object protobuf message]({{< ref "plugins/supported-languages/rich-plugins/rich-plugins-data-structures#object" >}}) with the `hook_name` property set to `MyPostMiddleware` and the `hook_type` property set to `Post`. This message will then then be dispatched to the gRPC server for processing before the request is sent upstream. - -
-{{< note success >}} -**Note** - -Ensure the plugin driver is configured as type *grpc*. Tyk will issue a request to your gRPC server for each plugin hook that you have configured. -{{< /note >}} - -#### Tyk Operator - -The examples below illustrate how to configure plugin hooks for an API Definition within Tyk Operator. - -Setting the `driver` configuring parameter to `gRPC` instructs Tyk Gateway to issue a request to your gRPC server for each plugin hook that you have configured. - -**Pre plugin hook example** - -In this example we can see that a `custom_middleware` configuration block has been used to configure a gRPC Pre request plugin hook with name `HelloFromPre`. Before any middleware is executed Tyk Gateway will serialize the request into a [Object protobuf message]({{< ref "plugins/supported-languages/rich-plugins/rich-plugins-data-structures#object" >}}) with the `hook_name` property set to `HelloFromPre` and the `hook_type` property set to `Pre`. This message will then then be dispatched to the gRPC server. - -```yaml {linenos=table,hl_lines=["14-18"],linenostart=1} -apiVersion: tyk.tyk.io/v1alpha1 -kind: ApiDefinition -metadata: - name: httpbin-grpc-pre -spec: - name: httpbin-grpc-pre - use_keyless: true - protocol: http - active: true - proxy: - target_url: http://httpbin.default.svc:8000 - listen_path: /httpbin-grpc-pre - strip_listen_path: true - custom_middleware: - driver: grpc - pre: - - name: HelloFromPre - path: "" -``` - -**Post plugin hook example** - -In the example we can see that a `custom_middleware` configuration block has been used to configure a gRPC Post plugin with name `HelloFromPost`. - -Before the request is sent upstream Tyk Gateway will serialize the request and session details into a [Object protobuf message]({{< ref "plugins/supported-languages/rich-plugins/rich-plugins-data-structures#object" >}}) with the `hook_name` property set to `HelloFromPost` and the `hook_type` property set to `Post`. This message will then then be dispatched to the gRPC server for processing before the request is sent upstream. - -```yaml {linenos=table,hl_lines=["14-18"],linenostart=1} -apiVersion: tyk.tyk.io/v1alpha1 -kind: ApiDefinition -metadata: - name: httpbin-grpc-post -spec: - name: httpbin-grpc-post - use_keyless: true - protocol: http - active: true - proxy: - target_url: http://httpbin.default.svc:8000 - listen_path: /httpbin-grpc-post - strip_listen_path: true - custom_middleware: - driver: grpc - post: - - name: HelloFromPost - path: "" -``` - -### Remote - -It is possible to configure your API so that it downloads a bundled configuration of your plugins from an external webserver. The bundled plugin configuration is contained within a zip file. - -A gRPC plugin bundle is similar to the [standard bundling mechanism]({{< ref "plugins/how-to-serve-plugins/plugin-bundles" >}}). The standard bundling mechanism zips the configuration and plugin source code, which will be executed by Tyk. Conversely, a gRPC plugin bundle contains only the configuration (`manifest.json`), with plugin code execution being handled independently by the gRPC server. - -Bundling a gRPC plugin requires the following steps: -- Create a `manifest.json` that contains the configuration of your plugins -- Build a zip file that bundles your plugin -- Upload the zip file to an external secured webserver -- Configure your API to download your plugin bundle - -#### Create the manifest file - -The `manifest.json` file specifies the configuration for your gRPC plugins. An example `manifest.json` is listed below: - -```yaml -{ - "file_list": [], - "custom_middleware": { - "pre": [{"name": "MyPreMiddleware"}], - "post": [{"name": "MyPostMiddleware"}], - "auth_check": {"name": "MyAuthCheck"}, - "driver": "grpc" - }, - "checksum": "", - "signature": "" -} -``` - -{{< note sucess >}} -**Note** - -The source code files, *file_list*, are empty for gRPC plugins. Your gRPC server contains the source code for handling plugins. -{{< /note >}} - -#### Build plugin bundle - -A plugin bundle can be built using the Tyk Gateway binary and should only contain the `manifest.json` file: - -```bash -tyk bundle build -output mybundle.zip -key mykey.pem -``` - -The example above generates a zip file, name `mybundle.zip`. The zip file is signed with key `mykey.pem`. - -The resulting bundle file should then be uploaded to the webserver that hosts your plugin bundles. - -#### Configure API - -##### Tyk Gateway - -To add a gRPC plugin to your API definition, you must specify the bundle file name within the `custom_middleware_bundle` field: - -```yaml -{ - "name": "Tyk Test API", - ... -+ "custom_middleware_bundle": "mybundle.zip" -} -``` - -The value of the `custom_middleware_bundle` field will be used in combination with the gateway settings to construct a bundle URL. For example, if Tyk Gateway is configured with a webserver base URL of https://my-bundle-server.com/bundles/ then an attempt would be made to download the bundle from https://my-bundle-server.com/bundles/mybundle.zip. - -##### Tyk Operator - - Currently this feature is not yet documented with a Tyk Operator example for configuring an API to use plugin bundles. For further details please reach out and contact us on the [community support forum](https://community.tyk.io). - ---- - -## Test your API Endpoint - -It is crucial to ensure the security and reliability of your gRPC server. As the developer, it is your responsibility to verify that your gRPC server is secured and thoroughly tested with appropriate test coverage. Consider implementing unit tests, integration tests and other testing methodologies to ensure the robustness of your server's functionality and security measures. This step ensures that the Tyk Gateway properly communicates with your gRPC server and executes the custom logic defined by the plugin hooks. - -Test the API endpoint using tools like *Curl* or *Postman*. Ensure that your gRPC server is running and the gRPC plugin(s) are functioning. An example using *Curl* is listed below: - -```bash -curl -X GET https://www.your-gateway-server.com:8080/api/path -``` - -Replace `https://www.your-gateway-server.com:8080/api/path` with the actual endpoint of your API. - ---- - -## Summary - -This guide has explained the key concepts and processes for writing gRPC plugins that integrate with Tyk Gateway. The following explanations have been given: - -- Prerequisites for developing a gRPC server for your target language. -- The *Dispatcher* service interface. -- How to configure Tyk Gateway to integrate with your gRPC server. -- How to configure Tyk Gateway with an optional external web server for fetching plugin configuration. -- How to configure gRPC plugins for your APIs. -- How to test your API integration with your gRPC server using curl. - ---- - -## What's Next? - -- Consult the [Protocol Buffer messages]({{< ref "plugins/supported-languages/rich-plugins/rich-plugins-data-structures" >}}) that Tyk Gateway uses when making a request to a gRPC server. -- Visit tutorial guides that explain how to implement a [Java]({{< ref "plugins/supported-languages/rich-plugins/grpc/request-transformation-java" >}}), [.NET]({{< ref "plugins/supported-languages/rich-plugins/grpc/custom-auth-dot-net" >}}) and [NodeJS]({{< ref "plugins/supported-languages/rich-plugins/grpc/custom-auth-nodejs" >}}) gRPC server. -- Visit our [plugins hub]({{< ref "plugins/plugin-hub" >}}) to explore further gRPC development examples and resources. - ---- diff --git a/tyk-docs/content/plugins/supported-languages/rich-plugins/luajit.md b/tyk-docs/content/plugins/supported-languages/rich-plugins/luajit.md deleted file mode 100755 index 8f32e14167..0000000000 --- a/tyk-docs/content/plugins/supported-languages/rich-plugins/luajit.md +++ /dev/null @@ -1,31 +0,0 @@ ---- -date: 2017-03-24T13:39:37Z -title: LuaJIT -menu: - main: - parent: "Rich Plugins" -weight: 0 -aliases: - - "plugins/supported-languages/rich-plugins/luajit" - - plugins/rich-plugins/luajit ---- -### Requirements - -Tyk uses [LuaJIT](http://luajit.org/). The main requirement is the LuaJIT shared library, you may find this as `libluajit-x` in most distros. - -For Ubuntu 14.04 you may use: - -`$ apt-get install libluajit-5.1-2 -$ apt-get install luarocks` - -The LuaJIT required modules are as follows: - -* [lua-cjson](https://github.com/mpx/lua-cjson): in case you have `luarocks`, run: `$ luarocks install lua-cjson` - -### How to write LuaJIT Plugins - -We have a demo plugin hosted in the repo [tyk-plugin-demo-lua](https://github.com/TykTechnologies/tyk-plugin-demo-lua). The project implements a simple middleware for header injection, using a Pre hook (see [Tyk custom middleware hooks]({{< ref "plugins/supported-languages/javascript-middleware/middleware-scripting-guide" >}})) and [mymiddleware.lua](https://github.com/TykTechnologies/tyk-plugin-demo-lua/blob/master/mymiddleware.lua). -### Lua Performance -Lua support is currently in beta stage. We are planning performance optimizations for future releases. -### Tyk Lua API Methods -Tyk Lua API methods aren’t currently supported. diff --git a/tyk-docs/content/plugins/supported-languages/rich-plugins/luajit/requirements.md b/tyk-docs/content/plugins/supported-languages/rich-plugins/luajit/requirements.md deleted file mode 100755 index f43cc7b1ba..0000000000 --- a/tyk-docs/content/plugins/supported-languages/rich-plugins/luajit/requirements.md +++ /dev/null @@ -1,23 +0,0 @@ ---- -date: 2017-03-24T13:43:51Z -title: Requirements -menu: - main: - parent: "LuaJIT" -weight: 0 -aliases: - - "plugins/supported-languages/rich-plugins/luajitrequirements" - - plugins/rich-plugins/luajit/requirements ---- - -Tyk uses [LuaJIT](http://luajit.org/). The main requirement is the LuaJIT shared library, you may find this as `libluajit-x` in most distros. - -For Ubuntu 14.04 you may use: - -`$ apt-get install libluajit-5.1-2 -$ apt-get install luarocks` - -The LuaJIT required modules are as follows: - -* [lua-cjson](https://github.com/mpx/lua-cjson): in case you have `luarocks`, run: `$ luarocks install lua-cjson` -* From v1.3.6 You can also use override response code, headers and body using ReturnOverrides. See the [Extend ReturnOverides](https://github.com/TykTechnologies/tyk/pull/763) sample for details. diff --git a/tyk-docs/content/plugins/supported-languages/rich-plugins/luajit/tutorial-add-demo-plugin-api.md b/tyk-docs/content/plugins/supported-languages/rich-plugins/luajit/tutorial-add-demo-plugin-api.md deleted file mode 100755 index 249930ba9e..0000000000 --- a/tyk-docs/content/plugins/supported-languages/rich-plugins/luajit/tutorial-add-demo-plugin-api.md +++ /dev/null @@ -1,91 +0,0 @@ ---- -date: 2017-03-24T13:46:17Z -title: Lua Plugin Tutorial -tags: ["Lua", "Lua tutorial"] -description: " A guide demonstrating the process of running a Lua plugin with Tyk Gateway" -menu: - main: - parent: "LuaJIT" -weight: 0 -aliases: - - plugins/supported-languages/rich-plugins/luajittutorial-add-demo-plugin-api - - plugins/rich-plugins/luajit/tutorial-add-demo-plugin-api ---- - -## Settings in the API Definition - -To add a Lua plugin to your API, you must specify the bundle name using the `custom_middleware_bundle` field: - -```json -{ - "name": "Tyk Test API", - "api_id": "1", - "org_id": "default", - "definition": { - "location": "header", - "key": "version" - }, - "auth": { - "auth_header_name": "authorization" - }, - "use_keyless": true, - "version_data": { - "not_versioned": true, - "versions": { - "Default": { - "name": "Default", - "expires": "3000-01-02 15:04", - "use_extended_paths": true, - "extended_paths": { - "ignored": [], - "white_list": [], - "black_list": [] - } - } - } - }, - "proxy": { - "listen_path": "/quickstart/", - "target_url": "http://httpbin.org", - "strip_listen_path": true - }, - "custom_middleware_bundle": "test-bundle", -} -``` - -## Global settings - -To enable Lua plugins you need to add the following block to `tyk.conf`: - -```json -"coprocess_options": { - "enable_coprocess": true, -}, -"enable_bundle_downloader": true, -"bundle_base_url": "http://my-bundle-server.com/bundles/", -"public_key_path": "/path/to/my/pubkey", -``` - -`enable_coprocess` enables the rich plugins feature. - -`enable_bundle_downloader` enables the bundle downloader. - -`bundle_base_url` is a base URL that will be used to download the bundle, in this example we have "test-bundle" specified in the API settings, Tyk will fetch the following URL: `http://my-bundle-server.com/bundles/test-bundle`. - -`public_key_path` sets a public key, this is used for verifying signed bundles, you may omit this if unsigned bundles are used. - -## Running the Tyk Lua build - -To use Tyk with Lua support you will need to use an alternative binary, it is provided in the standard Tyk package but it has a different service name. - -Firstly stop the standard Tyk version: - -```console -service tyk-gateway stop -``` - -and then start the Lua build: - -```console -service tyk-gateway-lua start -``` diff --git a/tyk-docs/content/plugins/supported-languages/rich-plugins/python/custom-auth-python-tutorial.md b/tyk-docs/content/plugins/supported-languages/rich-plugins/python/custom-auth-python-tutorial.md deleted file mode 100644 index cc47445c5e..0000000000 --- a/tyk-docs/content/plugins/supported-languages/rich-plugins/python/custom-auth-python-tutorial.md +++ /dev/null @@ -1,218 +0,0 @@ ---- -date: 2017-03-24T13:16:21Z -title: Custom Authentication Plugin Tutorial -menu: - main: - parent: "Python" -weight: 1 -aliases: - - /customise-tyk/plugins/rich-plugins/python/custom-auth-python-tutorial/ - - "plugins/supported-languages/rich-plugins/python/custom-auth-python-tutorial" - - plugins/rich-plugins/python/custom-auth-python-tutorial ---- - -## Introduction -This tutorial will guide you through the creation of a custom authentication plugin, written in Python. -A custom authentication plugin allows you to implement your own authentication logic and override the default Tyk authentication mechanism. The sample code implements a very simple key check; currently it supports a single, hard-coded key. It could serve as a starting point for your own authentication logic. We have tested this plugin with Ubuntu 14. - -The code used in this tutorial is also available in [this GitHub repository](https://github.com/TykTechnologies/tyk-plugin-demo-python). - -## Requirements - -* Tyk API Gateway: This can be installed using standard package management tools like Yum or APT, or from source code. See [here]({{< ref "tyk-self-managed#installation-options-for-tyk-self-managed" >}}) for more installation options. - -### Dependencies - -* The Tyk CLI utility, which is bundled with our RPM and DEB packages, and can be installed separately from [https://github.com/TykTechnologies/tyk-cli](https://github.com/TykTechnologies/tyk-cli) -* In Tyk 2.8 the Tyk CLI is part of the gateway binary, you can find more information by running "tyk help bundle". -* Python 3.4 - -## Create the Plugin -The first step is to create a new directory for your plugin file: - -```bash -mkdir ~/my-tyk-plugin -cd ~/my-tyk-plugin -``` - -Next you need to create a manifest file. This file contains information about our plugin file structure and how you expect it to interact with the API that will load it. -This file should be named `manifest.json` and needs to contain the following content: - -```json -{ - "file_list": [ - "middleware.py" - ], - "custom_middleware": { - "driver": "python", - "auth_check": { - "name": "MyAuthMiddleware" - } - } -} -``` - -* The `file_list` block contains the list of files to be included in the bundle, the CLI tool expects to find these files in the current working directory. -* The `custom_middleware` block contains the middleware settings like the plugin driver we want to use (`driver`) and the hooks that our plugin will expose. You use the `auth_check` for this tutorial. For other hooks see [here]({{< ref "plugins/supported-languages/rich-plugins/rich-plugins-work#coprocess-dispatcher---hooks" >}}). -* The `name` field references the name of the function that you implement in your plugin code: `MyAuthMiddleware`. -* You add an additional file called `middleware.py`, this will contain the main implementation of our middleware. - -{{< note success >}} -**Note** - -Your bundle should always contain a file named `middleware.py` as this is the entry point file. -{{< /note >}} - -### Contents of middleware.py - -You import decorators from the Tyk module as this gives you the `Hook` decorator, and you import [Tyk Python API helpers]({{< ref "plugins/supported-languages/rich-plugins/python/tyk-python-api-methods" >}}) - -You implement a middleware function and register it as a hook, the input includes the request object, the session object, the API meta data and its specification: - -```python -from tyk.decorators import * -from gateway import TykGateway as tyk - -@Hook -def MyAuthMiddleware(request, session, metadata, spec): - auth_header = request.get_header('Authorization') - if auth_header == '47a0c79c427728b3df4af62b9228c8ae': - tyk.log("I'm logged!", "info") - tyk.log("Request body" + request.object.body, "info") - tyk.log("API config_data" + spec['config_data'], "info") - session.rate = 1000.0 - session.per = 1.0 - metadata["token"] = "47a0c79c427728b3df4af62b9228c8ae" - return request, session, metadata -``` - - -You can modify the `manifest.json` to add as many files as you want. Files that aren't listed in the `manifest.json` file will be ignored when building the plugin bundle. - -## Building the Plugin - -A plugin bundle is a packaged version of the plugin, it may also contain a cryptographic signature of its contents. The `-y` flag tells the Tyk CLI tool to skip the signing process in order to simplify the flow of this tutorial. For more information on the Tyk CLI tool, see [here]({{< ref "plugins/how-to-serve-plugins/plugin-bundles" >}}). - -You will use the Dockerised version of the Tyk CLI tool to bundle our package. - -First, export your Tyk Gateway version to a variable. -```bash -### THIS MUST MATCH YOUR TYK GATEWAY VERSION -$ IMAGETAG=v3.1.2 -``` - -Then run the following commands to generate a `bundle.zip` in your current directory: -```docker -$ docker run \ - --rm -w "/tmp" -v $(pwd):/tmp \ - --entrypoint "/bin/sh" -it \ - tykio/tyk-gateway:$IMAGETAG \ - -c '/opt/tyk-gateway/tyk bundle build -y' -``` - -**Success!** - -You should now have a `bundle.zip` file in the plugin directory. - -## Publishing the Plugin - -To allow Tyk access to the plugin bundle, you need to serve this file using a web server. For this tutorial we'll use the Python built-in HTTP server (check the official docs for additional information). This server listens on port 8000 by default. To start it use: - -`python3 -m http.server` - -When the server is started our current working directory is used as the web root path, this means that our `bundle.zip` file should be accessible from the following URL: - -`http://:8000/bundle.zip` - -The Tyk Gateway fetches and loads a plugin bundle during startup time and subsequent reloads. For updating plugins using the hot reload feature, you should use different plugin bundle names as you expect them to be used for versioning purposes, e.g. bundle-1, bundle-2, etc. -If a bundle already exists, Tyk will skip the download process and load the version that's already present. - -## Configure Tyk - -You will need to modify the Tyk global configuration file (`tyk.conf`) to use Python plugins. The following block should be present in this file: - -```json -"coprocess_options": { - "enable_coprocess": true, - "python_path_prefix": "/opt/tyk-gateway" -}, -"enable_bundle_downloader": true, -"bundle_base_url": "http://dummy-bundle-server.com/bundles/", -"public_key_path": "/path/to/my/pubkey" -``` - -### Options - -* `enable_coprocess`: This enables the plugin -* `python_path_prefix`: Sets the path to built-in Tyk modules, this will be part of the Python module lookup path. The value used here is the default one for most installations. -* `enable_bundle_downloader`: This enables the bundle downloader -* `bundle_base_url`: This is a base URL that will be used to download the bundle. You should replace the `bundle_base_url` with the appropriate URL of the web server that's serving your plugin bundles. For now HTTP and HTTPS are supported but we plan to add more options in the future (like pulling directly from S3 buckets). You use the URL that's exposed by the test HTTP server in the previous step. -* `public_key_path`: Modify `public_key_path` in case you want to enforce the cryptographic check of the plugin bundle signatures. If the `public_key_path` isn't set, the verification process will be skipped and unsigned plugin bundles will be loaded normally. - -## Configure an API Definition - -There are two important parameters that you need to add or modify in the API definition. -The first one is `custom_middleware_bundle` which must match the name of the plugin bundle file. If we keep this with the default name that the Tyk CLI tool uses, it will be `bundle.zip`. - -`"custom_middleware_bundle": "bundle.zip"` - -The second parameter is specific to this tutorial, and should be used in combination with `use_keyless` to allow an API to authenticate against our plugin: - -`"use_keyless": false` -`"enable_coprocess_auth": true` - -`"enable_coprocess_auth"` will instruct the Tyk gateway to authenticate this API using the associated custom authentication function that's implemented by the plugin. - -## Configuration via the Tyk Dashboard - -To attach the plugin to an API, From the **Advanced Options** tab in the **API Designer** enter **bundle.zip** in the **Plugin Bundle ID** field. - -{{< img src="/img/2.10/plugin_bundle_id.png" alt="Plugin Options" >}} - -You also need to modify the authentication mechanism that's used by the API. -From the **Core Settings** tab in the **API Designer** select **Use Custom Authentication (Python, CoProcess, and JSVM plugins)** from the **Authentication - Authentication Mode** drop-down list. - -{{< img src="/img/2.10/custom_auth_python.png" alt="Advanced Options" >}} - -## Testing the Plugin - -Now you can simply make an API call against the API for which we've loaded the Python plugin. - - -### If Running Tyk Gateway from Source - -At this point you have your test HTTP server ready to serve the plugin bundle and the configuration with all the required parameters. -The final step is to start or restart the **Tyk Gateway** (this may vary depending on how you setup Tyk). -A separate service is used to load the Tyk version that supports Python (`tyk-gateway-python`), so we need to stop the standard one first (`tyk-gateway`): - -```service -service tyk-gateway stop -service tyk-gateway-python start -``` - -From now on you should use the following command to restart the service: - -```service -service tyk-gateway-python restart -``` - -A cURL request will be enough for testing our custom authentication middleware. - -This request will trigger a bad authentication: - -```curl -curl http://:8080/my-api/my-path -H 'Authorization: badtoken' -``` - -This request will trigger a successful authentication. You are using the token that's set by your Python plugin: - -```curl -curl http://:8080/my-api/my-path -H 'Authorization: 47a0c79c427728b3df4af62b9228c8ae' -``` - -## What's Next? - -In this tutorial you learned how Tyk plugins work. For a production-level setup we suggest the following steps: - -* Configure Tyk to use your own key so that you can enforce cryptographic signature checks when loading plugin bundles, and sign your plugin bundles! -* Configure an appropriate web server and path to serve your plugin bundles. diff --git a/tyk-docs/content/plugins/supported-languages/rich-plugins/python/performance.md b/tyk-docs/content/plugins/supported-languages/rich-plugins/python/performance.md deleted file mode 100755 index 0edc9c12e1..0000000000 --- a/tyk-docs/content/plugins/supported-languages/rich-plugins/python/performance.md +++ /dev/null @@ -1,17 +0,0 @@ ---- -date: 2017-03-24T13:11:14Z -title: Python Performance -menu: - main: - parent: "Python" -weight: 2 -aliases: - - "plugins/supported-languages/rich-plugins/python/performance" - - plugins/rich-plugins/python/performance ---- - -These are some benchmarks performed on Python plugins. Python plugins run in a standard Python interpreter, embedded inside Tyk. - -{{< img src="/img/diagrams/pythonResponseTime.png" alt="Python Performance" >}} - -{{< img src="/img/diagrams/pythonHitRate.png" alt="Python Performance" >}} diff --git a/tyk-docs/content/plugins/supported-languages/rich-plugins/python/python.md b/tyk-docs/content/plugins/supported-languages/rich-plugins/python/python.md deleted file mode 100755 index 552ddfd7a7..0000000000 --- a/tyk-docs/content/plugins/supported-languages/rich-plugins/python/python.md +++ /dev/null @@ -1,144 +0,0 @@ ---- -date: 2023-12-13T10:10:22Z -title: Python -menu: - main: - parent: "Rich Plugins" -weight: 4 -aliases: - - /customise-tyk/plugins/rich-plugins/rich-plugins-work/ - - /customise-tyk/plugins/rich-plugins/python/ - - /plugins/rich-plugins/python ---- - -### Requirements - -Since v2.9, Tyk supports any currently stable [Python 3.x version](https://www.python.org/downloads/). The main requirement is to have the Python shared libraries installed. These are available as `libpython3.x` in most Linux distributions. - -- Python3-dev -- [Protobuf](https://pypi.org/project/protobuf/): provides [Protocol Buffers](https://developers.google.com/protocol-buffers/) support -- [gRPC](https://pypi.org/project/grpcio/): provides [gRPC](http://www.grpc.io/) support - -### Important Note Regarding Performance - -Python plugins are [embedded](https://docs.python.org/3/extending/embedding.html) within the Tyk Gateway process. Tyk Gateway integrates with Python custom plugins via a [cgo](https://golang.org/cmd/cgo) bridge. - -`Tyk Gateway` <-> CGO <-> `Python Custom Plugin` - -In order to integrate with Python custom plugins, the *libpython3.x.so* shared object library is used to embed a Python interpreter directly in the Tyk Gateway. Further details can be found [here]({{< ref "plugins/supported-languages/rich-plugins/rich-plugins-work#coprocess-gateway-api" >}}) - -This allows combining the strengths of both Python and Go in a single application. However, it's essential to be aware of the potential complexities and performance implications of mixing languages, as well as the need for careful memory management when working with Python objects from Go. - -The Tyk Gateway process initialises the Python interpreter using [Py_initialize](https://docs.python.org/3/c-api/init.html#c.Py_Initialize). The Python [Global Interpreter Lock (GIL)](https://docs.python.org/3/glossary.html/#term-global-interpreter-lock) allows only one thread to execute Python bytecode at a time, ensuring thread safety and simplifying memory management. While the GIL simplifies these aspects, it can limit the scalability of multi-threaded applications, particularly those with CPU-bound tasks, as it restricts parallel execution of Python code. - -In the context of custom Python plugins, API calls are queued and the Python interpreter handles requests sequentially, processing them one at a time. Subsequently, this would consume large amounts of memory, and network sockets would remain open and blocked until the API request is processed. - -### Install the Python development packages - -{{< tabs_start >}} - -{{< tab_start "Docker" >}} -{{< note success >}} -**Note** - -Starting from Tyk Gateway version `v5.3.0`, Python is no longer bundled with the official Tyk Gateway Docker image by default, to address security vulnerabilities in the Python libraries highlighted by [Docker Scout](https://docs.docker.com/scout/). -
-Whilst Python plugins are still supported by Tyk Gateway, if you want to use them you must extend the image to add support for Python. For further details, please refer to the [release notes]({{< ref "developer-support/release-notes/gateway" >}}) for Tyk Gateway `v5.3.0`. -{{< /note >}} - -If you wish to use Python plugins using Docker, you can extend the official Tyk Gateway Docker image by adding Python to it. - -This example Dockerfile extends the official Tyk Gateway image to support Python plugins by installing python and the required modules: - -```dockerfile -ARG BASE_IMAGE -FROM ${BASE_IMAGE} AS base - -FROM python:3.11-bookworm -COPY --from=base /opt/tyk-gateway/ /opt/tyk-gateway/ -RUN pip install setuptools && pip install google && pip install 'protobuf==4.24.4' - -EXPOSE 8080 80 443 - -ENV PYTHON_VERSION=3.11 -ENV PORT=8080 - -WORKDIR /opt/tyk-gateway/ - -ENTRYPOINT ["/opt/tyk-gateway/tyk" ] -CMD [ "--conf=/opt/tyk-gateway/tyk.conf" ] -``` - -To use this, you simply run `docker build` with this Dockerfile, providing the Tyk Gateway image that you would like to extend as build argument `BASE_IMAGE`. -As an example, this command will extend Tyk Gateway `v5.3.0` to support Python plugins, generating the image `tyk-gateway-python:v5.3.0`: - -```bash -docker build --build-arg BASE_IMAGE=tykio/tyk-gateway:v5.3.0 -t tyk-gateway-python:v5.3.0 . -``` - -{{< tab_end >}} - -{{< tab_start "Ubuntu/Debian" >}} - -```apt -apt install python3 python3-dev python3-pip build-essential -``` - -### Install the Required Python Modules - -Make sure that "pip" is available in your system, it should be typically available as "pip", "pip3" or "pipX.X" (where X.X represents the Python version): - -```pip3 -pip3 install protobuf grpcio -``` - -{{< tab_end >}} - -{{< tab_start "Red Hat or CentOS" >}} - -```yum -yum install python3-devel python3-setuptools -python3 -m ensurepip -``` - -### Install the Required Python Modules - -Make sure that "pip" is now available in your system, it should be typically available as "pip", "pip3" or "pipX.X" (where X.X represents the Python version): - -```pip3 -pip3 install protobuf grpcio -``` - -{{< tab_end >}} - -{{< tabs_end >}} - -### Python versions - -Newer Tyk versions provide more flexibility when using Python plugins, allowing the users to set which Python version to use. By default, Tyk will try to use the latest version available. - -To see the Python initialisation log, run the Tyk gateway in debug mode. - -To use a specific Python version, set the `python_version` flag under `coprocess_options` in the Tyk Gateway configuration file (tyk.conf). - -{{< note success >}} -**Note** - -Tyk doesn't support Python 2.x. -{{< /note >}} - -### Troubleshooting - -To verify that the required Python Protocol Buffers module is available: - -```python3 -python3 -c 'from google import protobuf' -``` - -No output is expected from this command on successful setups. - -### How do I write Python Plugins? - -We have created [a demo Python plugin repository](https://github.com/TykTechnologies/tyk-plugin-demo-python). - -The project implements a simple middleware for header injection, using a Pre hook (see [Tyk custom middleware hooks]({{< ref "plugins/supported-languages/rich-plugins/rich-plugins-work#coprocess-dispatcher---hooks" >}}). A single Python script contains the code for it, see [middleware.py](https://github.com/TykTechnologies/tyk-plugin-demo-python/blob/master/middleware.py). diff --git a/tyk-docs/content/plugins/supported-languages/rich-plugins/python/tutorial-add-demo-plugin-api.md b/tyk-docs/content/plugins/supported-languages/rich-plugins/python/tutorial-add-demo-plugin-api.md deleted file mode 100755 index 90d8637a64..0000000000 --- a/tyk-docs/content/plugins/supported-languages/rich-plugins/python/tutorial-add-demo-plugin-api.md +++ /dev/null @@ -1,76 +0,0 @@ ---- -date: 2017-03-24T13:17:31Z -title: Tutorial - Add Python Plugin To Your Gateway -menu: - main: - parent: "Python" -weight: 3 -aliases: - - "plugins/supported-languages/rich-plugins/python/tutorial-add-demo-plugin-api" - - plugins/rich-plugins/python/tutorial-add-demo-plugin-api ---- - -## API settings - -To add a Python plugin to your API, you must specify the bundle name using the `custom_middleware_bundle` field: - -```{.json} -{ - "name": "Tyk Test API", - "api_id": "1", - "org_id": "default", - "definition": { - "location": "header", - "key": "version" - }, - "auth": { - "auth_header_name": "authorization" - }, - "use_keyless": true, - "version_data": { - "not_versioned": true, - "versions": { - "Default": { - "name": "Default", - "expires": "3000-01-02 15:04", - "use_extended_paths": true, - "extended_paths": { - "ignored": [], - "white_list": [], - "black_list": [] - } - } - } - }, - "proxy": { - "listen_path": "/quickstart/", - "target_url": "http://httpbin.org", - "strip_listen_path": true - }, - "custom_middleware_bundle": "test-bundle" -} -``` - -## Global settings - -To enable Python plugins you need to add the following block to `tyk.conf`: - -```{.copyWrapper} -"coprocess_options": { - "enable_coprocess": true, - "python_path_prefix": "/opt/tyk-gateway" -}, -"enable_bundle_downloader": true, -"bundle_base_url": "http://dummy-bundle-server.com/bundles/", -"public_key_path": "/path/to/my/pubkey", -``` - -`enable_coprocess`: enables the rich plugins feature. - -`python_path_prefix`: Sets the path to built-in Tyk modules, this will be part of the Python module lookup path. The value used here is the default one for most installations. - -`enable_bundle_downloader`: enables the bundle downloader. - -`bundle_base_url`: is a base URL that will be used to download the bundle, in this example we have `test-bundle` specified in the API settings, Tyk will fetch the URL for your specified bundle server (in the above example): `dummy-bundle-server.com/bundles/test-bundle`. You need to create and then specify your own bundle server URL. - -`public_key_path`: sets a public key, this is used for verifying signed bundles, you may omit this if unsigned bundles are used. diff --git a/tyk-docs/content/plugins/supported-languages/rich-plugins/python/tyk-python-api-methods.md b/tyk-docs/content/plugins/supported-languages/rich-plugins/python/tyk-python-api-methods.md deleted file mode 100755 index 00eb1ddc0d..0000000000 --- a/tyk-docs/content/plugins/supported-languages/rich-plugins/python/tyk-python-api-methods.md +++ /dev/null @@ -1,34 +0,0 @@ ---- -date: 2017-03-24T13:16:21Z -title: Tyk Python API methods -menu: - main: - parent: "Python" -weight: 1 -aliases: - - /customise-tyk/plugins/rich-plugins/python/tutorial-add-demo-plugin-api/ - - "plugins/supported-languages/rich-plugins/python/tyk-python-api-methods" - - plugins/rich-plugins/python/tyk-python-api-methods ---- - -Python plugins may call these Tyk API methods: - -### store_data(key, value, ttl) - -`store_data` sets a Redis `key` with the specified `value` and `ttl`. - -### get_data(key) - -`get_data` retrieves a Redis `key`. - -### trigger_event(event_name, payload) - -`trigger_event` triggers an internal Tyk event, the `payload` must be a JSON object. - -### log(msg, level) - -`log` will log a message (`msg`) using the specified `level`. - -### log_error(*args) - -`log_error` is a shortcut for `log`, it uses the error log level. diff --git a/tyk-docs/content/plugins/supported-languages/rich-plugins/rich-plugins-data-structures.md b/tyk-docs/content/plugins/supported-languages/rich-plugins/rich-plugins-data-structures.md deleted file mode 100644 index 60d7cfb1be..0000000000 --- a/tyk-docs/content/plugins/supported-languages/rich-plugins/rich-plugins-data-structures.md +++ /dev/null @@ -1,428 +0,0 @@ ---- -date: 2017-03-24T13:04:21Z -title: Rich Plugins Data Structures -menu: - main: - parent: "Rich Plugins" -weight: 3 -aliases: - - /plugins/rich-plugins/rich-plugins-data-structures ---- - -This page describes the data structures used by Tyk rich plugins, for the following plugin drivers: - -- Python (built-in) -- Lua (built-in) -- gRPC (external, compatible with any supported [gRPC language](https://grpc.io/docs/)) - -The Tyk [Protocol Buffer definitions](https://github.com/TykTechnologies/tyk/tree/master/coprocess/proto) are intended for users to generate their own bindings using the appropriate gRPC tools for the required target language. -The remainder of this document illustrates a class diagram and explins the attributes of the protobuf messages. - ---- - -## Class Diagram - -The class diagram below illustrates the structure of the [Object](#object) message, dispatched by Tyk to a gRPC server that handles custom plugins. - -{{< img src="/img/grpc/grpc-class-diagram.svg" width="600" >}} - ---- - -## Object - -The `Coprocess.Object` data structure wraps a `Coprocess.MiniRequestObject` and `Coprocess.ResponseObject` It contains additional fields that are useful for users that implement their own request dispatchers, like the middleware hook type and name. -It also includes the session state object (`SessionState`), which holds information about the current key/user that's used for authentication. - -```protobuf -message Object { - HookType hook_type = 1; - string hook_name = 2; - MiniRequestObject request = 3; - SessionState session = 4; - map metadata = 5; - map spec = 6; - ResponseObject response = 7; -} -``` - -#### Field Descriptions - -`hook_type` -Contains the middleware hook type: pre, post, custom auth. - -`hook_name` -Contains the hook name. - -`request` -Contains the request object, see `MiniRequestObject` for more details. - -`session` -Contains the session object, see `SessionState` for more details. - -`metadata` -Contains the metadata. This is a dynamic field. - -`spec` -Contains information about API definition, including `APIID`, `OrgID` and `config_data`. - -`response` -Contains information populated from the upstream HTTP response data, for response hooks. See [ResponseObject](#responseobject) for more details. All the field contents can be modified. - ---- - -## MiniRequestObject - -The `Coprocess.MiniRequestObject` is the main request data structure used by rich plugins. It's used for middleware calls and contains important fields like headers, parameters, body and URL. A `MiniRequestObject` is part of a `Coprocess.Object`. - -```protobuf -message MiniRequestObject { - map headers = 1; - map set_headers = 2; - repeated string delete_headers = 3; - string body = 4; - string url = 5; - map params = 6; - map add_params = 7; - map extended_params = 8; - repeated string delete_params = 9; - ReturnOverrides return_overrides = 10; - string method = 11; - string request_uri = 12; - string scheme = 13; - bytes raw_body = 14; -} -``` - -#### Field Descriptions - -`headers` -A read-only field for reading headers injected by previous middleware. Modifying this field won't alter the request headers See `set_headers` and `delete_headers` for this. - -`set_headers` -This field appends the given headers (keys and values) to the request. - -`delete_headers` -This field contains an array of header names to be removed from the request. - -`body` -Contains the request body. See `ReturnOverrides` for response body modifications. - -`raw_body` -Contains the raw request body (bytes). - -`url` -The request URL. - -`params` -A read-only field that contains the request params. Modifying this value won't affect the request params. - -`add_params` -Add paramaters to the request. - -`delete_params` -This field contains an array of parameter keys to be removed from the request. - -`return_overrides` -See `ReturnOverrides` for more information. - -`method` -The request method, e.g. GET, POST, etc. - -`request_uri` -Raw unprocessed URL which includes query string and fragments. - -`scheme` -Contains the URL scheme, e.g. `http`, `https`. - ---- - -## ResponseObject - -The `ResponseObject` exists within an [object](#object) for response hooks. The fields are populated with the upstream HTTP response data. All the field contents can be modified. - -```protobuf -syntax = "proto3"; - -package coprocess; - -message ResponseObject { - int32 status_code = 1; - bytes raw_body = 2; - string body = 3; - map headers = 4; - repeated Header multivalue_headers = 5; -} - -message Header { - string key = 1; - repeated string values = 2; -} -``` - -#### Field Descriptions - -`status_code` -This field indicates the HTTP status code that was sent by the upstream. - -`raw_body` -This field contains the HTTP response body (bytes). It's always populated. - -`body` -This field contains the HTTP response body in string format. It's not populated if the `raw_body` contains invalid UTF-8 characters. - -`headers` -A map that contains the headers sent by the upstream. - -`multivalue_headers` -A list of headers, each header in this list is a structure that consists of two parts: a key and its corresponding values. -The key is a string that denotes the name of the header, the values are a list of strings that hold the content of the header, this is useful when the header has multiple associated values. -This field is available for Go, Python and Ruby since tyk v5.0.4 and 5.1.1+. - ---- - -## ReturnOverrides - -The `ReturnOverrides` object, when returned as part of a `Coprocess.Object`, overrides the response of a given HTTP request. It also stops the request flow and the HTTP request isn't passed upstream. The fields specified in the `ReturnOverrides` object are used as the HTTP response. -A sample usage for `ReturnOverrides` is when a rich plugin needs to return a custom error to the user. - -```protobuf -syntax = "proto3"; - -package coprocess; - -message ReturnOverrides { - int32 response_code = 1; - string response_error = 2; - map headers = 3; - bool override_error = 4; - string response_body = 5; -} -``` - -#### Field Descriptions - -`response_code` -This field overrides the HTTP response code and can be used for error codes (403, 500, etc.) or for overriding the response. - -`response_error` -This field overrides the HTTP response body. - -`headers` -This field overrides response HTTP headers. - -`override_error` -This setting provides enhanced customization for returning custom errors. It should be utilized alongside `response_body` for optimal effect. - -`response_body` -This field serves as an alias for `response_erro`r and holds the HTTP response body. - ---- - -## SessionState {#session-state} - -A `SessionState` data structure is created for every authenticated request and stored in Redis. It's used to track the activity of a given key in different ways, mainly by the built-in Tyk middleware like the quota middleware or the rate limiter. -A rich plugin can create a `SessionState` object and store it in the same way built-in authentication mechanisms do. This is what a custom authentication middleware does. This is also part of a `Coprocess.Object`. -Returning a null session object from a custom authentication middleware is considered a failed authentication and the appropriate HTTP 403 error is returned by the gateway (this is the default behavior) and can be overridden by using `ReturnOverrides`. - -#### Field Descriptions - -`last_check` -No longer used. - -`allowance` -No longer in use, should be the same as `rate`. - -`rate` -The number of requests that are allowed in the specified rate limiting window. - -`per` -The number of seconds that the rate window should encompass. - -`expires` -An epoch that defines when the key should expire. - -`quota_max` -The maximum number of requests allowed during the quota period. - -`quota_renews` -An epoch that defines when the quota renews. - -`quota_remaining` -Indicates the remaining number of requests within the user's quota, which is independent of the rate limit. - -`quota_renewal_rate` -The time in seconds during which the quota is valid. So for 1000 requests per hour, this value would be 3600 while `quota_max` and `quota_remaining` would be 1000. - -`access_rights` -Defined as a `map` instance, that maps the session's API ID to an [AccessDefinition](#access-definition). The AccessDefinition defines the [access rights]({{< ref "api-management/policies#setting-granular-paths-on-a-per-key-basis" >}}) for the API in terms of allowed: versions and URLs(endpoints). Each URL (endpoint) has a list of allowed methods. For further details consult the tutorials for how to create a [security policy]({{< ref "getting-started/create-security-policy" >}}) for Tyk Cloud, Tyk Self Managed and Tyk OSS platforms. - -`org_id` -The organization this user belongs to. This can be used in conjunction with the org_id setting in the API Definition object to have tokens "owned" by organizations. - -`oauth_client_id` -This is set by Tyk if the token is generated by an OAuth client during an OAuth authorization flow. - -`basic_auth_data` -This section contains a hashed representation of the basic auth password and the hashing method used. -For further details see [BasicAuthData](#basicauthdata). - -`jwt_data` -Added to sessions where a Tyk key (embedding a shared secret) is used as the public key for signing the JWT. The JWT token's KID header value references the ID of a Tyk key. See [JWTData](#jwtdata) for an example. - -`hmac_enabled` -When set to `true` this indicates generation of a [HMAC signature]({{< ref "/api-management/client-authentication#sign-requests-with-hmac" >}}) using the secret provided in `hmac_secret`. If the generated signature matches the signature provided in the *Authorization* header then authentication of the request has passed. - -`hmac_secret` -The value of the HMAC shared secret. - -`is_inactive` -Set this value to true to deny access. - -`apply_policy_id` -The policy ID that is bound to this token. - -{{< note success >}} -**Note** - -Although `apply_policy_id` is still supported, it is now deprecated. `apply_policies` is now used to list your policy IDs as an array. This supports the **[Multiple Policy]({{< ref "api-management/policies#partitioned-policy-functionality" >}})** feature introduced in the **v2.4 - 1.4** release. -{{< /note >}} - -`data_expires` -A value, in seconds, that defines when data generated by this token expires in the analytics DB (must be using Pro edition and MongoDB). - -`monitor` -Defines a [quota monitor]({{< ref "basic-config-and-security/report-monitor-trigger-events/monitors" >}}) containing a list of percentage threshold limits in descending order. These limits determine when webhook notifications are triggered for API users or an organization. Each threshold represents a percentage of the quota that, when reached, triggers a notification. See [Monitor](#monitor) for further details and an example. - -`enable_detailed_recording` -Set this value to true to have Tyk store the inbound request and outbound response data in HTTP Wire format as part of the analytics data. - -`metadata` -Metadata to be included as part of the session. This is a key/value string map that can be used in other middleware such as transforms and header injection to embed user-specific data into a request, or alternatively to query the providence of a key. - -`tags` -Tags are embedded into analytics data when the request completes. If a policy has tags, those tags will supersede the ones carried by the token (they will be overwritten). - -`alias` -As of v2.1, an Alias offers a way to identify a token in a more human-readable manner, add an Alias to a token in order to have the data transferred into Analytics later on so you can track both hashed and un-hashed tokens to a meaningful identifier that doesn't expose the security of the underlying token. - -`last_updated` -A UNIX timestamp that represents the time the session was last updated. Applicable to *Post*, *PostAuth* and *Response* plugins. When developing *CustomAuth* plugins developers should add this to the SessionState instance. - -`id_extractor_deadline` -This is a UNIX timestamp that signifies when a cached key or ID will expire. This relates to custom authentication, where authenticated keys can be cached to save repeated requests to the gRPC server. See [id_extractor]({{< ref "plugins/plugin-types/auth-plugins/id-extractor" >}}) and [Auth Plugins]({{< ref "plugins/plugin-types/auth-plugins/auth-plugins" >}}) for additional information. - -`session_lifetime` -UNIX timestamp that denotes when the key will automatically expire. Any·subsequent API request made using the key will be rejected. Overrides the global session lifetime. See [Key Expiry and Deletion]({{< ref "/api-management/client-authentication#set-physical-key-expiry-and-deletion" >}}) for more information. - ---- - -## AccessDefinition {#access-definition} - -```protobuf -message AccessDefinition { - string api_name = 1; - string api_id = 2; - repeated string versions = 3; - repeated AccessSpec allowed_urls = 4; -} -``` - -Defined as an attribute within a [SessionState](#session-state) instance. Contains the allowed versions and URLs (endpoints) for the API that the session request relates to. Each URL (endpoint) specifies an associated list of allowed methods. See also [AccessSpec](#access-spec). - -#### Field Descriptions - -`api_name` -The name of the API that the session request relates to. - -`api_id` -The ID of the API that the session request relates to. - -`versions` -List of allowed API versions, e.g. `"versions": [ "Default" ]`. - -`allowed_urls` List of [AccessSpec](#access-spec) instances. Each instance defines a URL (endpoint) with an associated allowed list of methods. If all URLs (endpoints) are allowed then the attribute is not set. - ---- - -## AccessSpec {#access-spec} - -Defines an API's URL (endpoint) and associated list of allowed methods - -```protobuf -message AccessSpec { - string url = 1; - repeated string methods = 2; -} -``` - -#### Field Descriptions - -`url` -A URL (endpoint) belonging to the API associated with the request session. - -`methods` -List of allowed methods for the URL (endpoint), e.g. `"methods": [ "GET". "POST", "PUT", "PATCH" ]`. - ---- - -## BasicAuthData - -The `BasicAuthData` contains a hashed password and the name of the hashing algorithm used. This is represented by the `basic_auth_data` attribute in [SessionState](#session-state) message. - -```yaml -"basicAuthData": { - "password": , - "hash": -} -``` - -#### Field Descriptions - -`password` -A hashed password. - -`hash` -Name of the [hashing algorithm]({{< ref "api-management/policies#access-key-hashing" >}}) used to hash the password. - ---- - -## JWTData - -Added to [sessions](#session-state) where a Tyk key (embedding a shared secret) is used as the public key for signing the JWT. This message contains the shared secret. - -```yaml -"jwtData": { - "secret": "the_secret" -} -``` - -#### Field Descriptions - -`secret` -The shared secret. - ---- - -## Monitor {#monitor} -Added to a [session](#session-state) when [monitor quota thresholds]({{< ref "basic-config-and-security/report-monitor-trigger-events/monitors" >}}) are defined within the Tyk key. This message contains the quota percentage threshold limits, defined in descending order, that trigger webhook notification. - -```yaml -message Monitor { - repeated double trigger_limits = 1; -} -``` - -#### Field Descriptions - -`trigger_limits` -List of trigger limits defined in descending order. Each limit represents the percentage of the quota that must be reached in order for the webhook notification to be triggered. - -```yaml -"monitor": { - "trigger_limits": [80.0, 60.0, 50.0] -} -``` - ---- - -
\ No newline at end of file diff --git a/tyk-docs/content/plugins/supported-languages/rich-plugins/rich-plugins-work.md b/tyk-docs/content/plugins/supported-languages/rich-plugins/rich-plugins-work.md deleted file mode 100755 index 4a3a099780..0000000000 --- a/tyk-docs/content/plugins/supported-languages/rich-plugins/rich-plugins-work.md +++ /dev/null @@ -1,153 +0,0 @@ ---- -date: 2017-03-24T13:04:21Z -title: How do rich plugins work ? -menu: - main: - parent: "Rich Plugins" -weight: 2 -aliases: - - /plugins/rich-plugins/rich-plugins-work ---- - -### ID Extractor & Auth Plugins - -The ID Extractor is a caching mechanism that's used in combination with Tyk Plugins. It can be used specifically with plugins that implement custom authentication mechanisms. The ID Extractor works for all rich plugins: gRPC-based plugins, Python and Lua. - -See [ID Extractor]({{< ref "plugins/plugin-types/auth-plugins/id-extractor" >}}) for more details. - -### Interoperability - -This feature implements an in-process message passing mechanism, based on [Protocol Buffers](https://developers.google.com/protocol-buffers/), any supported languages should provide a function to receive, unmarshal and process this kind of messages. - -The main interoperability task is achieved by using [cgo](https://golang.org/cmd/cgo/) as a bridge between a supported language -like Python- and the Go codebase. - -Your C bridge function must accept and return a `CoProcessMessage` data structure like the one described in [`api.h`](https://github.com/TykTechnologies/tyk/blob/master/coprocess/api.h), where `p_data` is a pointer to the serialised data and `length` indicates the length of it. - -```{.copyWrapper} -struct CoProcessMessage { - void* p_data; - int length; -}; -``` - -The unpacked data will hold the actual `CoProcessObject` data structure. - -- `HookType` - the hook type (see below) -- `Request` - the HTTP request -- `Session` - the [Tyk session object]({{< ref "tyk-apis/tyk-gateway-api/token-session-object-details" >}}). -- `Metadata` - the metadata from the session data above (key/value string map). -- `Spec` - the API specification data. Currently organization ID, API ID and config_data. - -```{.copyWrapper} -type CoProcessObject struct { - HookType string - Request CoProcessMiniRequestObject - Session SessionState - Metadata map[string]string - Spec map[string]string -} -``` - -### Coprocess Dispatcher - -`Coprocess.Dispatcher` describes a very simple interface for implementing the dispatcher logic, the required methods are: `Dispatch`, `DispatchEvent` and `Reload`. - -`Dispatch` accepts a pointer to a `struct CoProcessObject` (as described above) and must return an object of the same type. This method will be called for every configured hook on every request. Traditionally this method will perform a single function call on the target language side (like `Python_DispatchHook` in `coprocess_python`), and the corresponding logic will be handled from there (mostly because different languages have different ways of loading, referencing or calling middlewares). - -`DispatchEvent` provides a way of dispatching Tyk events to a target language. This method doesn't return any variables but does receive a JSON-encoded object containing the event data. For extensibility purposes, this method doesn't use Protocol Buffers, the input is a `[]byte`, the target language will take this (as a `char`) and perform the JSON decoding operation. - -`Reload` is called when triggering a hot reload, this method could be useful for reloading scripts or modules in the target language. - -### Coprocess Dispatcher - Hooks - -This component is in charge of dispatching your HTTP requests to the custom middlewares. The list, from top to bottom, shows the order of execution. The dispatcher follows the standard middleware chain logic and provides a simple mechanism for "hooking" your custom middleware behavior, the supported hooks are: - -* **Pre**: gets executed before the request is sent to your upstream target and before any authentication information is extracted from the header or parameter list of the request. When enabled, this applies to both keyless and protected APIs. -* **AuthCheck**: gets executed as a custom authentication middleware, instead of the standard ones provided by Tyk. Use this to provide your own authentication mechanism. -* **PostKeyAuth**: gets executed right after the authentication process. -* **Post**: gets executed after the authentication, validation, throttling, and quota-limiting middleware has been executed, just before the request is proxied upstream. Use this to post-process a request before sending it to your upstream API. This is only called when using protected APIs. If you want to call a hook after the authentication but before the validation, throttling and other middleware, see **PostKeyAuth**. -* **Response**: gets executed after the upstream API replies. The arguments passed to this hook include both the request and response data. Use this to modify the HTTP response before it's sent to the client. This hook also receives the request object, the session object, the metadata and API definition associated with the request. - -{{< note success >}} -**Note** - -Response hooks are not available for native Go plugins. Python and gRPC plugins are supported. -{{< /note >}} - - -### Coprocess Gateway API - -[`coprocess_api.go`](https://github.com/TykTechnologies/tyk/tree/master/coprocess) provides a bridge between the Gateway API and C. Any function that needs to be exported should have the `export` keyword: - -```{.copyWrapper} -//export TykTriggerEvent -func TykTriggerEvent( CEventName *C.char, CPayload *C.char ) { - eventName := C.GoString(CEventName) - payload := C.GoString(CPayload) - - FireSystemEvent(tykcommon.TykEvent(eventName), EventMetaDefault{ - Message: payload, - }) -} -``` - -You should also expect a header file declaration of this function in [`api.h`](https://github.com/TykTechnologies/tyk/blob/master/coprocess/api.h), like this: - -```{.copyWrapper} -#ifndef TYK_COPROCESS_API -#define TYK_COPROCESS_API -extern void TykTriggerEvent(char* event_name, char* payload); -#endif -``` - -The language binding will include this header file (or declare the function inline) and perform the necessary steps to call it with the appropriate arguments (like an FFI mechanism could do). As a reference, this is how this could be achieved if you're building a [Cython](http://cython.org/) module: - -```{.copyWrapper} -cdef extern: - void TykTriggerEvent(char* event_name, char* payload); - -def call(): - event_name = 'my event'.encode('utf-8') - payload = 'my payload'.encode('utf-8') - TykTriggerEvent( event_name, payload ) -``` - -### Basic usage - -The intended way of using a Coprocess middleware is to specify it as part of an API Definition: - -```{.json} -"custom_middleware": { - "pre": [ - { - "name": "MyPreMiddleware", - "require_session": false - }, - { - "name": "AnotherPreMiddleware", - "require_session": false - } - ], - "post": [ - { - "name": "MyPostMiddleware", - "require_session": false - } - ], - "post_key_auth": [ - { - "name": "MyPostKeyAuthMiddleware", - "require_session": true - } - ], - "auth_check": { - "name": "MyAuthCheck" - }, - "driver": "python" -} -``` -{{< note success >}} -**Note** - -All hook types support chaining except the custom auth check (`auth_check`). -{{< /note >}} diff --git a/tyk-docs/content/plugins/tutorials/quick-starts/go/dashboard.md b/tyk-docs/content/plugins/tutorials/quick-starts/go/dashboard.md deleted file mode 100644 index a49d9f90f0..0000000000 --- a/tyk-docs/content/plugins/tutorials/quick-starts/go/dashboard.md +++ /dev/null @@ -1,108 +0,0 @@ ---- -title: Dashboard Plugins Quickstart -description: Explains how to build and run the getting started example within Tyk Dashboard -tags: ["custom", "plugin", "plugins", "go", "goplugins", "go plugin", "tyk go plugin", "golang plugin"] ---- - - -This quick start explains how to run the [getting started](https://github.com/TykTechnologies/custom-go-plugin) Go plugin within Tyk Dashboard. - -**Estimated time**: 10-15 minutes - -In this tutorial you will learn how to: - -1. Add your Tyk license. -2. Bootstrap the Tyk Dashboard environment. -3. Login to Tyk Dashboard. -4. View the pre-configured API. -5. Test the plugin. -6. View the analytics. -7. Next steps. - -## 1. Add your Tyk license - -Create and edit the file `.env` with your Tyk Dashboard license key - -```console -# Make a copy of the example .env file for the Tyk-Dashboard -cp .env.example .env -``` - -## 2. Bootstrap the getting started example - -run the `make` command: - -```bash -make -``` - -This will take a few minutes to run as it compiles the plugin for the first time and downloads all the necessary Docker images. - -## 3. Log in to Tyk Dashboard - -Log on to the Tyk Dashboard on `http://localhost:3000` using the following Bootstrapped credentials: -``` -demo@tyk.io -``` -and password: -``` -topsecretpassword -``` - -Note: these are editable in `.env.example` - -## 4. View the pre-configured API - -Once you're logged on to the Tyk Dashboard, navigate to the *APIs* screen. - -You'll see a sample *Httpbin* API. Let's click into it for more details. - -Click on *VIEW RAW DEFINITION*. Note the *custom_middleware* block is filled out, injecting the compiled example Go plugin into the API. - -## 5. Test the plugin - -Let's send an API request to the API Gateway so it can reverse proxy to our API. - -```terminal -curl localhost:8080/httpbin/get -``` - -Yields the response: -``` -{ - "args": {}, - "headers": { - "Accept": "*/*", - "Accept-Encoding": "gzip", - "Foo": "Bar", - "Host": "httpbin.org", - "User-Agent": "curl/7.79.1", - "X-Amzn-Trace-Id": "Root=1-63f78c47-51e22c5b57b8576b1225984a" - }, - "origin": "172.26.0.1, 99.242.70.243", - "url": "http://httpbin.org/get" -} -``` - -Note, we see a *Foo:Bar* HTTP Header was injected by our Go plugin and echoed back to us by the Httpbin mock server. - -## 6. View the analytics - -Navigate to the Dashboard's various *API Usage Data* to view analytics on the API request! - -## 7. Next steps - -Try updating the code of the plugin and experimenting. Once you've made changes to the example plugin, please run `make build` to compile the plugin and reload the gateway with the changes. - -When finished, please run `make down` to bring down the stack. - -## Summary - -This tutorial has explained how to: -1. Add your Tyk license. -2. Bootstrap the Tyk Dashboard environment. -3. Login to Tyk Dashboard. -4. View the pre-configured API. -5. Test the plugin. -6. View the analytics. -7. Next steps. diff --git a/tyk-docs/content/plugins/tutorials/quick-starts/go/open-source.md b/tyk-docs/content/plugins/tutorials/quick-starts/go/open-source.md deleted file mode 100644 index 7314e86633..0000000000 --- a/tyk-docs/content/plugins/tutorials/quick-starts/go/open-source.md +++ /dev/null @@ -1,87 +0,0 @@ ---- -title: Open-Source Plugins Quickstart -description: Explains how to build and run the getting started example using Tyk OSS Gateway -tags: ["custom", "plugin", "plugins", "go", "goplugins", "go plugin", "tyk go plugin", "golang plugin"] ---- - - -This quick start guide will explain how to run the [getting started](https://github.com/TykTechnologies/custom-go-plugin) Go plugin using the Tyk OSS Gateway. - -**Estimated time**: 10-15 minutes - -In this tutorial you will learn how to: - -1. Bootstrap the getting started example. -2. Test the plugin. -3. View the analytics. -4. Next steps. - -## 1. Bootstrap the getting started example - -Please run the following command from within your newly cloned directory to run the Tyk Stack and compile the sample plugin. This will take a few minutes as we have to download all the necessary dependencies and docker images. - -```bash -make up-oss && make build -``` - -## 2. Test the plugin - -Let's test the plugin by sending an API request to the pre-configured API definition: - -``` -curl localhost:8080/httpbin/get -``` - -Response: -``` -{ - "args": {}, - "headers": { - "Accept": "*/*", - "Accept-Encoding": "gzip", - "Foo": "Bar", - "Host": "httpbin.org", - "User-Agent": "curl/7.79.1" - }, - "origin": "172.28.0.1, 99.242.70.243", - "url": "http://httpbin.org/get" -} -``` - -We've sent an API request to the Gateway. We can see that the sample custom plugin has injected an HTTP header with a value of *Foo:Bar*. This header was echoed back in the Response Body via the mock Httpbin server. - -The `./tyk/scripts/bootstrap-oss.sh` script creates an API definition that includes the custom plugin. - - -## 3. View the analytics - -We can see that Tyk Pump is running in the background. Let's check the logs after sending the API request: - -``` -docker logs custom-go-plugin_tyk-pump_1 -``` - -Output: -``` -time="Feb 23 16:29:27" level=info msg="Purged 1 records..." prefix=stdout-pump -{"level":"info","msg":"","time":"0001-01-01T00:00:00Z","tyk-analytics-record":{"method":"GET","host":"httpbin.org","path":"/get","raw_path":"/get","content_length":0,"user_agent":"curl/7.79.1","day":23,"month":2,"year":2023,"hour":16,"response_code":200,"api_key":"00000000","timestamp":"2023-02-23T16:29:27.53328605Z","api_version":"Non Versioned","api_name":"httpbin","api_id":"845b8ed1ae964ea5a6eccab6abf3f3de","org_id":"","oauth_id":"","request_time":1128,"raw_request":"...","raw_response":"...","ip_address":"192.168.0.1","geo":{"country":{"iso_code":""},"city":{"geoname_id":0,"names":null},"location":{"latitude":0,"longitude":0,"time_zone":""}},"network":{"open_connections":0,"closed_connections":0,"bytes_in":0,"bytes_out":0},"latency":{"total":1128,"upstream":1111},"tags":["key-00000000","api-845b8ed1ae964ea5a6eccab6abf3f3de"],"alias":"","track_path":false,"expireAt":"2023-03-02T16:29:27.54271855Z","api_schema":""}} -``` - -As we can see, when we send API requests, the Tyk Pump will scrape them from Redis and then send them to a persistent store as configured in the Tyk Pump env file. - -In this example, we've configured a simple `STDOUT` Pump where the records will be printed to the Standard OUT (docker logs!) - -## 4. Next steps - -Try updating the code of the plugin and experimenting. Once you've made changes to the example plugin, please run `make build` to compile the plugin and reload the gateway with the changes. - -When finished, please run `make down` to bring down the stack. - - -## Summary - -This tutorial has explained how to: -1. Bootstrap the getting started example plugin in Tyk Gateway environment. -2. Test the example plugin. -3. View the analytics. -4. Next steps. diff --git a/tyk-docs/content/plugins/tutorials/quick-starts/go/quickstart.md b/tyk-docs/content/plugins/tutorials/quick-starts/go/quickstart.md deleted file mode 100644 index 0f9f6f962e..0000000000 --- a/tyk-docs/content/plugins/tutorials/quick-starts/go/quickstart.md +++ /dev/null @@ -1,51 +0,0 @@ ---- -date: 2017-03-24T15:45:13Z -description: Explains purpose and prerequisites for running quickstart Go plugin -title: Go Plugins Quickstart -tags: ["custom", "plugin", "plugins", "go", "goplugins", "go plugin", "tyk go plugin", "golang plugin"] -aliases: - - /plugins/get-started-selfmanaged/deploy - - /plugins/get-started-selfmanaged/get-started - - /plugins/get-started-selfmanaged/run - - /plugins/get-started-selfmanaged/test - - /plugins/get-started-plugins ---- - -This section takes you through the process of running and building a quickstart Go plugin, included within Tyk's [getting started](https://github.com/TykTechnologies/custom-go-plugin) repository. Go plugins are the recommended plugin type and suitable for most use cases. - -## Expected outcome - -At the end of this process you should have a Tyk Gateway or Tyk Self-Managed environment running locally, with a simple Go plugin executing on each API request. For each reponse to an API request the example plugin will inject a *Foo* header, with a value of *Bar*. - -## Prerequisites - -- [Docker](https://docs.docker.com/get-docker/) -- [Docker-compose](https://docs.docker.com/compose/install/) -- [Tyk license](https://tyk.io/sign-up/#self) (if using Self-Managed Tyk, which will make the process easier via UI) -- [Make](https://www.gnu.org/software/make) -- OSX (Intel) -> Not a prerequisite, though these steps are tested on OSX Intel/ARM - -## Before you begin - -Please clone the [getting started](https://github.com/TykTechnologies/custom-go-plugin) respository. - -```bash -git clone https://github.com/TykTechnologies/custom-go-plugin -cd custom-go-plugin -``` - -## Choose your environment - -{{< grid >}} -{{< badge read="15 mins" imageStyle="object-fit:contain" href="plugins/tutorials/quick-starts/go/dashboard" image="/img/logos/tyk-logo-selfmanaged.svg" alt="Dashboard">}} -Dashboard Tutorial -{{< /badge >}} - -{{< badge read="15 mins" imageStyle="object-fit:contain" href="plugins/tutorials/quick-starts/go/open-source" image="/img/logos/tyk-logo-selfmanaged.svg" alt="Tyk OSS Gateway">}} -Tyk OSS Gateway Tutorial -{{< /badge >}} -{{< /grid >}} - -## Next Steps - -Try our advanced Go Plugin [tutorials]({{< ref "tyk-cloud#configure-plugins" >}}). diff --git a/tyk-docs/content/product-stack/tyk-gateway/advanced-configurations/plugins/api-config/classic.md b/tyk-docs/content/product-stack/tyk-gateway/advanced-configurations/plugins/api-config/classic.md deleted file mode 100644 index 2b1bf3cbf2..0000000000 --- a/tyk-docs/content/product-stack/tyk-gateway/advanced-configurations/plugins/api-config/classic.md +++ /dev/null @@ -1,125 +0,0 @@ ---- -date: 2024-06-25T12:59:42Z -title: Configuring Plugins for Tyk Classic APIs -description: "This section explains how to configure Tyk Classic APIs to use plugin source code co-located on Tyk Gateway server" -tags: [ "Tyk Classic plugins" ] ---- - -An API can be configured so that one or more of its associated plugins can execute at different phases of the request / response lifecycle. Each plugin configuration serves to identify the plugin source file path and the name of the corresponding function, triggered at each request / response lifecycle stage. - -This guide explains how to configure plugins for Tyk Classic APIs within the [Tyk Classic API definition](#tyk-classic-apidef) or via the [API designer](#tyk-classic-dashboard) in Tyk Dashboard. - -If you’re using the newer Tyk OAS APIs, then check out the [Tyk OAS]({{< ref "/product-stack/tyk-gateway/advanced-configurations/plugins/api-config/oas" >}}) page. - ---- - -## Configuring plugins in the Tyk Classic API Definition {#tyk-classic-apidef} - -In Tyk Classic APIs, the *custom_middleware* section of the Tyk Classic API Definition is where you configure plugins that will run at different points during the lifecycle of an API request. - -This table illustrates the different phases of the API request lifecycle where custom plugins can be executed: - -| Phase | Description | Config | -| ----- | --- | ---- | -| Pre | Executed at the start of the request processing chain | `pre` | -| Auth | Executed during the authentication step | `auth_check` | -| Post Auth | Executed after the requester has been authenticated | `post_key_auth` | -| Post | Executed at the end of the request processing chain | `post` | -| Response | Executed on the response received from the upstream | `response` | - -This example configuration illustrates how to set up plugins for different phases of the request lifecycle: - -```json {linenos=true, linenostart=1} -{ - "custom_middleware": { - "pre": [ - { - "name": "PreHook1", - "path": "/path/to/plugin1.so", - "disabled": false, - "require_session": false, - "raw_body_only": false - } - ], - "auth_check": { - "name": "AuthCheck", - "path": "/path/to/plugin.so", - "disabled": false, - "require_session": false, - "raw_body_only": false - }, - "post_key_auth": [ - { - "name": "PostKeyAuth", - "path": "/path/to/plugin.so", - "disabled": false, - "require_session": false, - "raw_body_only": false - } - ], - "post": [ - { - "name": "PostHook1", - "path": "/path/to/plugin1.so", - "disabled": false, - "require_session": false, - "raw_body_only": false - }, - { - "name": "PostHook2", - "path": "/path/to/plugin2.so", - "disabled": false, - "require_session": false, - "raw_body_only": false - } - ], - "response": [ - { - "name": "ResponseHook", - "path": "/path/to/plugin.so", - "disabled": false, - "require_session": false, - "raw_body_only": false - } - ], - "driver": "goplugin" - } -} -``` - -In this example we can see that there are Golang custom authentication (`auth_check`), post authentication (`post_key_auth`), post, pre and response plugins configured. - -It can be seen that each plugin is configured with the specific function name and associated source file path of the file that contains the function. Furthermore, each lifecycle phase (except `auth`) can have a list of plugins configured, allowing for complex processing workflows. For example, you might develop one plugin for logging and another for modifying the request in the pre request phase. When multiple plugins are configured for a phase they will be executed in the order that they appear in the API definition. - -The `driver` configuration parameter describes the plugin implementation language. Please refer to the [supported languages]({{< ref "/plugins/supported-languages#plugin-driver-names" >}}) section for list of supported plugin driver names. - -Each plugin can have additional settings, such as: -- `disabled`: When true, disables the plugin. -- `raw_body_only`: When true, indicates that only the raw body should be processed. -- `require_session`: When true, indicates that session metadata will be available to the plugin. This is applicable only for post, post authentication and response plugins. - ---- - -## Configuring plugins in the API Designer {#tyk-classic-dashboard} - -This section explains how to configure plugins for a Tyk Classic API using Tyk Dashboard. It specifically covers the use case where the source files of your plugins are deployed on the Tyk Gateway file system. - -Select your API from the list of *Created APIs* to reach the API designer and then follow these steps: - -{{< img src="/img/plugins/plugins_classic_api_source_config.png" alt="Plugins Classic API screen" >}} - -#### Step 1: Display the Tyk Classic API Definition editor - -Click on the **View Raw Definition** button to display an editor for updating the Tyk Classic API Definition. - -{{< img src="/img/plugins/plugins_classic_api_definition_editor.png" alt="Plugins Classic API Definition editor screen" >}} - -#### Step 2: Edit the Tyk Classic API Definition to configure plugins - -Use the editor to edit the `custom_middleware` section of the [Tyk Classic API Definition]({{< ref "/product-stack/tyk-gateway/advanced-configurations/plugins/api-config/classic" >}}). - -{{< img src="/img/plugins/plugins_classic_api_bundles_config.png" alt="Plugins Classic API Bundle Field" >}} - -#### Step 3: Save changes - -Select the **Update** button to apply your changes to the Tyk Classic API Definition. \ No newline at end of file diff --git a/tyk-docs/content/product-stack/tyk-gateway/advanced-configurations/plugins/api-config/oas.md b/tyk-docs/content/product-stack/tyk-gateway/advanced-configurations/plugins/api-config/oas.md deleted file mode 100644 index cbf1236f84..0000000000 --- a/tyk-docs/content/product-stack/tyk-gateway/advanced-configurations/plugins/api-config/oas.md +++ /dev/null @@ -1,159 +0,0 @@ ---- -date: 2024-06-25T12:59:42Z -title: Configuring Plugins for Tyk OAS APIs -description: "This section explains how to configure Tyk OAS APIs to use plugin bundles deployed on a remote web server" -tags: [ "Tyk OAS plugins" ] ---- - -An API can be configured so that one or more of its associated plugins can execute at different phases of the request / response life cycle. Each plugin configuration serves to identify the plugin source file path and the name of the corresponding function, triggered at each request / response lifecycle stage. - -This guide explains how to configure plugins for Tyk OAS APIs within the [Tyk OAS API definition](#tyk-oas-apidef) or via the [API designer](#tyk-oas-dashboard) in Tyk Dashboard. - -If you’re using the legacy Tyk Classic APIs, then check out the [Tyk Classic]({{< ref "product-stack/tyk-gateway/advanced-configurations/plugins/api-config/classic" >}}) page. - ---- - -## Configuring plugins in the Tyk OAS API Definition {#tyk-oas-apidef} - -The `x-tyk-api-gateway.middleware.global` section is used to configure plugins in a Tyk OAS API. It contains a `pluginConfig` section and a list of plugins for each phase of the API request / response lifecycle. - -The `pluginConfig` section contains the `driver` parameter that is used to configure the plugin implementation [language]({{< ref "/plugins/supported-languages#plugin-driver-names" >}}): - -```yaml -"pluginConfig": { - "driver": "goplugin" -} -``` - -Within the `x-tyk-api-gateway.middleware.global` section, keyed lists of plugins can be configured for each phase of the API request / response lifecycle described in the table below: - -| Phase | Description | Config Key | -| ----- | --- | ---- | -| Pre | Executed at the start of the request processing chain | `prePlugins` | -| Post Auth | Executed after the requester has been authenticated | `postAuthenticationPlugins` | -| Post | Executed at the end of the request processing chain | `postPlugins` | -| Response | Occurs after the main request processing but before the response is sent. | `responsePlugins` | - -Each plugin configuration can have the following fields configured: - -- `enabled`: When true, enables the plugin. -- `functionName`: The name of the function that implements the plugin within the source file. -- `path`: The path to the plugin source file. -- `rawBodyOnly`: When true, indicates that only the raw body should be processed. -- `requireSession`: When true, indicates that session metadata will be available to the plugin. This is applicable only for post, post authentication and response plugins. - -For example a Post Authentication plugin would be configured within a `postAuthenticationPlugins` list as shown below: - -```yaml -"postAuthenticationPlugins": [ - { - "enabled": true, - "functionName": "post_authentication_func", - "path": "/path/to/plugin1.so", - "rawBodyOnly": true, - "requireSession": true - } -] -``` - -An full example is given below to illustrate how to set up plugins for different phases of the request / response lifecycle: - -```json {linenos=true, linenostart=1, hl_lines=["15-52"]} -{ - "x-tyk-api-gateway": { - "info": { - "dbId": "667962397f6de50001508ac4", - "id": "b4d8ac6e5a274d7c7959d069b47dc206", - "orgId": "6672f4377f6de50001508abf", - "name": "OAS APIs Plugins", - "state": { - "active": true, - "internal": false - } - }, - "middleware": { - "global": { - "pluginConfig": { - "driver": "goplugin" - }, - "postAuthenticationPlugins": [ - { - "enabled": true, - "functionName": "post_authentication_func", - "path": "/path/to/plugin1.so", - "rawBodyOnly": true, - "requireSession": true - } - ], - "postPlugins": [ - { - "enabled": true, - "functionName": "postplugin", - "path": "/path/to/plugin1.so", - "rawBodyOnly": true, - "requireSession": true - } - ], - "prePlugins": [ - { - "enabled": true, - "functionName": "pre-plugin", - "path": "/path/to/plugin1.so" - } - ], - "responsePlugins": [ - { - "enabled": true, - "functionName": "Response", - "path": "/path/to/plugin1.so", - "rawBodyOnly": true, - "requireSession": true - } - ] - } - } - } -} -``` - -In this example we can see that the plugin driver has been configured by setting the `driver` field to `goplugin` within the `pluginConfig` object. This configuration instructs Tyk Gateway that our plugins are implemented using Golang. - -We can also see that the following type of plugins are configured: - -- **Pre**: A plugin is configured within the `prePlugins` list. The plugin is enabled and implemented by function `pre-plugin` within the source file located at path `/path/to/plugin1.so`. -- **Post Authentication**: A plugin is configured within the `postAuthenticationPlugins` list. The plugin is enabled and implemented by function `post_authentication_func` within the source file located at path `/path/to/plugin1.so`. The raw request body and session metadata is available to the plugin. -- **Post**: A plugin is configured within the `responsePlugins` list. The plugin is enabled and implemented by function `postplugin` within the source file located at path `/path/to/plugin1.so`. The raw request body and session metadata is available to the plugin. -- **Response**: A plugin is configured within the `postPlugins` list. The plugin is enabled and implemented by function `Response` within the source file located at path `/path/to/plugin1.so`. The raw request body and session metadata is available to the plugin. - -The configuration above is a complete and valid Tyk OAS API Definition that you can use as a basis for trying out custom plugins. You will need to update the [driver]({{< ref "plugins/supported-languages#plugin-driver-names" >}}) parameter to reflect the target language type of your plugins. You will also need to update the `path` and `functionName` parameters for each plugin to reflect the source code. - ---- - -## Configuring plugins in the API Designer {#tyk-oas-dashboard} - -Select your API from the list of *Created APIs* to reach the API designer and then follow these steps: - -#### Step 1: Configure plugin type and custom data - -In the *Plugins Configuration* section, select the *Plugin Driver*, which tells Tyk which type of plugin to expect: Go, gRPC, JavaScript (OTTO), Lua or Python. - -You can configure custom data that will be made available to your plugin function as a JSON formatted object in the *Config Data* option. - -{{< img src="/img/plugins/plugins_oas_api_driver_options.png" alt="OAS API Plugins Driver Config" >}} - - -#### Step 2: Configure the custom plugins - -For each plugin that you wish to register with the API, click on the **Add Plugin** button to display a plugin configuration section: - -{{< img src="/img/plugins/plugins_oas_api_source_config.png" alt="OAS Plugins Config Section" >}} - -Complete the following fields: - -- `Function Name`: Enter the name of the function within your plugin code that Tyk should invoke. -- `Path`: Enter the path to the source file that contains the function that implements your plugin. -- `Raw Body Only`: Optionally, toggle the *Raw Body Only* switch to true when you do not wish to fill body in request or response object for your plugins. - -#### Step 3: Save the API - -Select **Save API** to apply the changes to your API. \ No newline at end of file diff --git a/tyk-docs/content/product-stack/tyk-gateway/advanced-configurations/plugins/api-config/overview.md b/tyk-docs/content/product-stack/tyk-gateway/advanced-configurations/plugins/api-config/overview.md deleted file mode 100644 index 654866bbe3..0000000000 --- a/tyk-docs/content/product-stack/tyk-gateway/advanced-configurations/plugins/api-config/overview.md +++ /dev/null @@ -1,49 +0,0 @@ ---- -date: 2024-08-28 -title: Custom Plugins API Overview -description: "This section explains an overview for how to configure plugins for APIs" -tags: [ "plugins" ] ---- - -This page provides an overview on how to register one or more custom plugins to be executed at different stages or [hooks]({{< ref "plugins/plugin-types/plugintypes#plugin-and-hook-types" >}}) in the API request/response lifecycle. If you wish to learn how to register custom plugins to be executed on the traffic logs generated by the Gateway please refer to the [analytics plugins]({{< ref "plugins/plugin-types/analytics-plugins" >}}) page. - -If you need fine-grained control at the endpoint level then it is also possible to configure [per-endpoint plugins]({{< ref "product-stack/tyk-gateway/middleware/endpoint-plugin" >}}). These are custom Golang plugins that are triggered at the end of the request processing chain before API-level *Post* plugins are executed. - ---- - -## Introduction - -There are three locations where Tyk Gateway can find plugin functions: - -1. **gRPC plugins**: Plugin functions are implemented by a gRPC server with the associated configuration specified with the API definition. For further details on how to configure gRPC plugins, please refer to our [gRPC]({{< ref "plugins/supported-languages/rich-plugins/grpc" >}}) documentation. -2. **Local plugins**: Plugins are implemented by functions within source code files located on the Gateway's file system. The API Definition allows the source code file path and function name to be configured for each plugin. For further details read on. -3. **Plugin bundles**: The plugin source code and configuration are bundled into a zip file that is served by a remote web server. For further details see the [plugin bundles]({{< ref "plugins/how-to-serve-plugins/plugin-bundles" >}}) page. - -## Plugin configuration - -Each plugin for an API can be configured within the API Definition with the following details: - -| Property | Description | -|-------|-------------| -| `Enabled` | When true, the plugin is activated | -| `Name` | A name used to identify the plugin | -| `Path` | The path to the source code file on the Tyk Gateway file system | -| `Function name` | The name of the function that implements the plugin. The function should exist within the source code file referenced in `path` | -| `Raw body only` | When set to true, this flag indicates that only the raw request body should be processed | -| `Require session state`| When set to true, Tyk Gateway will serialize the request session state and pass it as an argument to the function that implements the plugin in the target language. This is applicable to Post, Response, and Authentication hooks only | - ---- - -## Language configuration - -For local and bundle plugins a [plugin driver]({{< ref "plugins/supported-languages#plugin-driver-names" >}}) is configured to specify the plugin implementation language. If using gRPC plugins a `grpc` plugin driver should be used to instruct Tyk to request execution of plugins from within a gRPC server that is external to the Tyk process. This offers additional language support since Tyk can integrate with a gRPC server that is implemented using any supported [gRPC language](https://grpc.io/docs/). - -For a given API it is not possible to mix the implementation language for the plugin types: Pre, Authentication, Post, Post Authentication and Response plugins. For example, it is not possible to implement a pre request plugin in *Go* and also implement a post request plugin in *Python* for the same API. - ---- - -## Next steps - -If you’re using the newer Tyk OAS APIs, then check out the [configuring plugins for Tyk OAS APis]({{< ref "/product-stack/tyk-gateway/advanced-configurations/plugins/api-config/oas" >}}) page for further details. - -If you’re using the legacy Tyk Classic APIs, then check out the [configuring plugins for Tyk Classic APIs]({{< ref "/product-stack/tyk-gateway/advanced-configurations/plugins/api-config/classic" >}}) page for further details. diff --git a/tyk-docs/content/product-stack/tyk-gateway/advanced-configurations/plugins/bundles/bundle-cli.md b/tyk-docs/content/product-stack/tyk-gateway/advanced-configurations/plugins/bundles/bundle-cli.md deleted file mode 100644 index e4617135c0..0000000000 --- a/tyk-docs/content/product-stack/tyk-gateway/advanced-configurations/plugins/bundles/bundle-cli.md +++ /dev/null @@ -1,253 +0,0 @@ ---- -date: 2024-08-20T13:32:12Z -title: Bundler CLI Tool -description: "Explains usage of the bundler CLI tool" -tags: ["Bundle CLI Tool"] ---- - -The bundler tool is a CLI service, provided by _Tyk Gateway_ as part of its binary since v2.8. This lets you generate -[plugin bundles]({{< ref "/plugins/how-to-serve-plugins/plugin-bundles" >}}). - -{{< note >}} -**Note** -Generated plugin bundles must be served using your own web server. -{{< /note >}} - -Issue the following command to see more details on the `bundle` command: - -```bash -/opt/tyk-gateway/bin/tyk bundle -h -``` - ---- - -## Prerequisites - -To create plugin bundles you will need the following: - -- **Manifest.json**: The [manifest.json]({{< ref "plugins/how-to-serve-plugins/plugin-bundles#manifest" >}}) file - contains the paths to the plugin source files and the name of the function implementing each plugin. The - _manifest.json_ file is mandatory and must exist on the Tyk Gateway file system. By default the bundle CLI looks for - a file named _manifest.json_ in the current working directory where the bundle command is run from. The exact location - can be specified using the `--manifest` command option. -- **Plugin source code files**: The plugin source code files should be contained relative to the directory in which the - _manifest.json_ file is located. The _manifest.json_ should contain relative path references to source code files. - - {{< note >}} - **Note** - Source code files are not required when creating a plugin bundle for gRPC plugins since the plugin - source code is located at the gRPC server. - {{< /note >}} - -- **Certificate key**: Plugin bundles can optionally be signed with an RSA private key. The corresponding public key - should be located in the file configured in environmental variable `TYK_GW_PUBLICKEYPATH` or the `public_key_path` - parameter in `tyk.conf`: - -```json -{ - "enable_bundle_downloader": true, - "bundle_base_url": "http://my-bundle-server.com/bundles/", - "public_key_path": "/path/to/my/pubkey.pem" -} -``` - ---- - -## Directory Structure - -A suggested directory structure is shown below for Golang, Javascript and Python bundles in the tabs below. - -{{< note success >}} -**Note** - -Sub-directories (folders) are not supported inside the `bundle-directory` location. - -{{< /note >}} - -{{< tabs_start >}} {{< tab_start "Golang" >}} - -```bash -/bundle-directory -├── manifest.json # Manifest file with plugin references -└── plugin.so # Compiled Golang plugin -``` - -{{< tab_end >}} {{< tab_start "Javascript" >}} - -```bash -/bundle-directory -├── manifest.json # Manifest file with plugin references -├── plugin1.js # First JavaScript plugin source file -└── plugin2.js # Second JavaScript plugin source file -``` - -{{< tab_end >}} - -{{< tab_start "Python" >}} - -```bash -/bundle-directory -├── manifest.json # Manifest file with plugin references -├── plugin1.py # First Python plugin source file -└── plugin2.py # Second Python plugin source file -``` - -{{< tab_end >}} - -{{< tabs_end >}} - -The `manifest.json` will reference the files located in the `bundle-directory`, ensure plugin source files are organized relative to the manifest. The Tyk Gateway will load and execute these plugins based on the paths defined in the `manifest.json` file. - -Sample `manifest.json` is shown below for Golang, Javascript and Python bundles in the tabs below. - -{{< tabs_start >}} {{< tab_start "Golang" >}} - -```json -{ - "file_list": [ - "plugin.so" - ], - "custom_middleware": { - "pre": [ - { - "name": "PreMiddleware", - "path": "./plugin.so" - } - ], - "post": [ - { - "name": "PostMiddleware", - "path": "./plugin.so" - } - ], - "driver": "goplugin" - }, - "checksum": "", - "signature": "" -} - -``` - -{{< tab_end >}} {{< tab_start "Javascript" >}} - -```json -{ - "file_list": [ - "plugin1.js", - "plugin2.js" - ], - "custom_middleware": { - "pre": [ - { - "name": "PreMiddleware", - "path": "./plugin1.js" - } - ], - "post": [ - { - "name": "PostMiddleware", - "path": "./plugin2.js" - } - ], - "driver": "otto" - }, - "checksum": "", - "signature": "" -} -``` - -{{< tab_end >}} - -{{< tab_start "Python" >}} - -```json -{ - "file_list": [ - "plugin1.py", - "plugin2.py" - ], - "custom_middleware": { - "pre": [ - { - "name": "PreMiddleware", - "path": "./plugin1.py" - } - ], - "post": [ - { - "name": "PostMiddleware", - "path": "./plugin2.py" - } - ], - "driver": "python" - }, - "checksum": "", - "signature": "" -} -``` - -{{< tab_end >}} - -{{< tabs_end >}} - ---- - -## Creating a plugin bundle - -Run the following command to create the bundle: - -```bash -$ tyk bundle build -``` - -The resulting file will contain all your specified files and a modified `manifest.json` with the checksum and signature -(if required) applied, in ZIP format. - -By default, Tyk will attempt to sign plugin bundles for improved security. If no private key is specified, the program -will prompt for a confirmation. Use `-y` to override this (see options below). - ---- - -## Command Options - -Instructions on how to create plugin bundles is displayed by issuing the following command: - -```bash -/opt/tyk-gateway/bin/tyk bundle build -h -``` - -The following options are supported: - -- `--manifest`: Specifies the path to the manifest file. This defaults to `manifest.json` within the current working - directory. -- `--output`: Specifies the name of the bundle file e.g. `--output bundle-latest.zip`. If this flag is not specified, - `bundle.zip` will be used. -- `-y`: Force tool to create unsigned bundle without prompting e.g. `$ tyk bundle build --output bundle-latest.zip -y`. -- `--key`: Specifies the path to your private key which is used to generate signed bundle e.g. - `$ tyk bundle build --output bundle-latest.zip --key=mykey.pem`. - ---- - -## Docker Example - -Since v5.5 Tyk Gateway uses distroless docker images. - -For Gateway version < v5.5 it is possible to use Docker to create plugin bundles as shown in the example below. - -```bash -docker run --rm -it \ - --name bundler \ - -v `pwd`:/plugin-source \ - -v `pwd`/../../../confs/keys:/keys \ - -w /plugin-source \ - --entrypoint /bin/bash \ - tykio/tyk-gateway:v5.4.0 \ - -c 'export PATH="/opt/tyk-gateway:$$PATH"; tyk bundle build -o bundle.zip -k /keys/key.pem' -``` - -This Docker command runs a container using the `tykio/tyk-gateway:v5.4.0` image to build a Tyk plugin bundle. It mounts -the current directory from the host as `/plugin-source` and a directory containing keys as `/keys` inside the container. -The working directory within the container is set to `/plugin-source`, and the default entrypoint is overridden to use -`/bin/bash`. The command executed in the container exports a modified `PATH` to include the Tyk Gateway binaries, then -runs `tyk bundle build` to generate a plugin bundle named `bundle.zip`, using the specified key for authentication. The -container is automatically removed after the command completes, and the operation is conducted interactively. diff --git a/tyk-docs/content/product-stack/tyk-gateway/advanced-configurations/plugins/bundles/classic.md b/tyk-docs/content/product-stack/tyk-gateway/advanced-configurations/plugins/bundles/classic.md deleted file mode 100644 index e9e4615dae..0000000000 --- a/tyk-docs/content/product-stack/tyk-gateway/advanced-configurations/plugins/bundles/classic.md +++ /dev/null @@ -1,77 +0,0 @@ ---- -date: 2024-06-25T12:59:42Z -title: Tyk Classic API Configuring Plugin Bundles -description: "This section explains how to configure Tyk Classic APIs to use plugin bundles deployed on a remote web server" -tags: ["Tyk plugins", "API Gateway middleware", "Custom middleware", "Custom API request", "Tyk Classic API"] ---- - -For custom plugins that are deployed as [plugin bundles]({{< ref "/plugins/how-to-serve-plugins/plugin-bundles" >}}), the API should be configured with the name of the plugin bundle file to download from your remote web server. Furthermore, the Gateway should be [configured]({{< ref "/plugins/how-to-serve-plugins/plugin-bundles#gateway-configuration" >}}) to enable downloading plugin bundles. - -You can configure your API with the name of the plugin bundle file to download within the Tyk Classic API definition or API Designer. - -If you’re using the newer Tyk OAS APIs, then check out the [Tyk OAS]({{< ref "/product-stack/tyk-gateway/advanced-configurations/plugins/bundles/oas" >}}) page. - -## Configuring Plugin Bundles in the Tyk Classic API Definition - -The configuration for an API to fetch and download a plugin bundle from a remote server is encapsulated within the `custom_middleware_bundle` field of the Tyk Classic API Definition. An illustrative example is listed below: - -```json {hl_lines=["33"], linenos=true, linenostart=1} -{ - "name": "Tyk Classic Bundle API", - "api_id": "1", - "org_id": "default", - "definition": { - "location": "header", - "key": "version" - }, - "auth": { - "auth_header_name": "authorization" - }, - "use_keyless": true, - "version_data": { - "not_versioned": true, - "versions": { - "Default": { - "name": "Default", - "expires": "3000-01-02 15:04", - "use_extended_paths": true, - "extended_paths": { - "ignored": [], - "white_list": [], - "black_list": [] - } - } - } - }, - "proxy": { - "listen_path": "/quickstart/", - "target_url": "http://httpbin.org", - "strip_listen_path": true - }, - "custom_middleware_bundle": "bundle-latest.zip" -} -``` - -With the configuration given in the example above, calls to the API will invoke the custom plugins defined in the `manifest.json` file contained within `bundle-latest.zip` uploaded to your remote webserver, e.g. `http://your-example-plugin-server.com/plugins`. - -Tyk Gateway should be configured for downloading plugin bundles from a secured web server. Please consult the [plugin bundles]({{< ref "/plugins/how-to-serve-plugins/plugin-bundles" >}}) documentation for further details. - ---- - -## Configuring plugin bundles in the API Designer - -To configure plugin bundles for Tyk Classic APIs click on the APIs menu item in the *API Management* menu of Dashboard and select your API to display the API editor screen. Subsequently, follow the steps below: - -##### Step 1: Access plugin options - -Click on the *Advanced Options* tab and scroll down until the *Plugin Options* section is displayed. - -{{< img src="/img/plugins/plugins_classic_api_bundles_config.png" alt="Tyk Classic Plugin Options section" >}} - -##### Step 2: Enter relative path to bundle file - -Enter the relative path of the plugin bundle file in the *Plugin Bundle ID* field that Tyk Gateway should download from the web server hosting plugin bundles. - -##### Step 3: Save the API - -Select the **save** or **update** button to apply the changes to your API. diff --git a/tyk-docs/content/product-stack/tyk-gateway/advanced-configurations/plugins/bundles/oas.md b/tyk-docs/content/product-stack/tyk-gateway/advanced-configurations/plugins/bundles/oas.md deleted file mode 100644 index 88c0b25c96..0000000000 --- a/tyk-docs/content/product-stack/tyk-gateway/advanced-configurations/plugins/bundles/oas.md +++ /dev/null @@ -1,102 +0,0 @@ ---- -date: 2024-08-20T12:59:42Z -title: Tyk OAS API Plugin Bundle Configuration -description: "This section explains how to configure Tyk OAS APIs to use plugin bundles" -tags: [ "Tyk plugins", "API Gateway middleware", "Custom middleware", "Custom API request", "Tyk OAS API" ] ---- - -For API plugins that are deployed as [plugin bundles]({{< ref "/plugins/how-to-serve-plugins/plugin-bundles" >}}), the API should be configured with the name of the plugin bundle file to download from your remote web server. Furthermore, the Gateway should be [configured]({{< ref "/plugins/how-to-serve-plugins/plugin-bundles#gateway-configuration" >}}) to enable downloading plugin bundles. - -You can configure your API with the name of the plugin bundle file to download within the Tyk OAS API definition or API Designer. - -If you’re using the legacy Tyk Classic APIs, then check out the [Tyk Classic]({{< ref "/product-stack/tyk-gateway/advanced-configurations/plugins/bundles/classic" >}}) page. - -## Configuring Plugin Bundles in the Tyk OAS API Definition - -The configuration for a Tyk OAS API to fetch the download of a plugin bundle from a remote web server is encapsulated within the `pluginConfig` section within the `middleware.global` section of the `x-tyk-api-gateway` part of a Tyk OAS API Definition. - -The `pluginConfig` section is structured as follows: - -- `bundle`: A JSON entity that contains the following configuration parameters: - - `enabled`: When `true`, enables the plugin. - - `path`: The relative path of the zip file in relation to the base URL configured on the remote webserver that hosts plugin bundles. -- `driver`: Indicates the type of plugin, e.g. `golang`, `grpc`, `lua`, `otto` or `python`. - -An illustrative example is listed below: - -```json{hl_lines=["37-45"], linenos=true, linenostart=1} -{ - "components": {}, - "info": { - "title": "example-oas-plugin-configuration", - "version": "1.0.0" - }, - "openapi": "3.0.3", - "paths": { - "/anything": { - "put": { - "operationId": "anythingput", - "responses": { - "200": { - "description": "" - } - } - } - } - }, - "x-tyk-api-gateway": { - "info": { - "name": "example-oas-plugin-configuration", - "state": { - "active": true - } - }, - "upstream": { - "url": "http://httpbin.org/" - }, - "server": { - "listenPath": { - "value": "/example-oas-plugin-configuration/", - "strip": true - } - }, - "middleware": { - "global": { - "pluginConfig": { - "bundle": { - "enabled": true, - "path": "plugin.zip" - }, - "driver": "goplugin" - } - } - } - } -} -``` - -In this example we can see that bundle plugin has been configured within the `middleware.global.pluginConfig.bundle` object. The plugin is enabled and bundled within file `plugin.zip`. The plugin bundle is a Go plugin, i.e. `middleware.global.pluginConfig.driver` has been configured with value `goplugin`. - -The configuration above is a complete and valid Tyk OAS API Definition that you can import into Tyk to try out custom plugin bundles, assuming that you have provided a valid bundle file named `plugin.zip`. - -## Configuring Plugin Bundles in the API Designer - -To configure plugin bundles for Tyk OAS APIs click on the APIs menu item in the *API Management* menu of Dashboard and select your API to display the editor screen. Subsequently, follow the steps below: - -##### Step 1: Access plugin options - -Scroll down until the *Enable Plugin* section is displayed. - -{{< img src="/img/plugins/plugins_oas_api_bundles_config.png" alt="Tyk OAS API Bundle section" >}} - -##### Step 2: Enable plugin bundle for you API - -Enable a plugin bundle for your API by activating the toggle switch. - -##### Step 3: Enter relative path to plugin bundle file - -Enter the relative path of the plugin bundle file in the *Plugin Bundle ID* field that Tyk Gateway should download from the web server that hosts your plugin bundles. - -##### Step 4: Save the API - -Select **Save API** to apply the changes to your API. \ No newline at end of file diff --git a/tyk-docs/content/product-stack/tyk-gateway/advanced-configurations/plugins/golang/go-development-flow.md b/tyk-docs/content/product-stack/tyk-gateway/advanced-configurations/plugins/golang/go-development-flow.md deleted file mode 100644 index 47c9146ac6..0000000000 --- a/tyk-docs/content/product-stack/tyk-gateway/advanced-configurations/plugins/golang/go-development-flow.md +++ /dev/null @@ -1,274 +0,0 @@ ---- -title: Custom Go plugin development flow -tags: - - custom plugin - - golang - - go plugin - - middleware - - debugging go plugins -description: Development flow working with Go Plugins -date: "2025-01-06" ---- - -Go Plugins need to be compiled to native shared object code, which can then be loaded by Tyk Gateway. - -We recommend that you familiarize yourself with the following official Go documentation to help you work effectively with Go plugins: - -- [The official plugin package documentation - Warnings](https://pkg.go.dev/plugin) -- [Tutorial: Getting started with multi-module workspaces](https://go.dev/doc/tutorial/workspaces) - -{{< note success >}} -**Note** - -Plugins are currently supported only on Linux, FreeBSD, and macOS, making them unsuitable for applications intended to be portable. -{{< /note >}} - -## Tyk Plugin Compiler - -We provide the [Tyk Plugin Compiler](https://tyk.io/docs/product-stack/tyk-gateway/advanced-configurations/plugins/golang/go-plugin-compiler/) docker image, which we **strongly recommend** is used to build plugins compatible with the official Gateway releases. That tool provides the cross compilation toolchain, Go version used to build the release, ensures that compatible flags are used when compiling plugins (such as `-trimpath`, `CC`, `CGO_ENABLED`, `GOOS`, `GOARCH`) and also works around known Go issues such as: - -- https://github.com/golang/go/issues/19004 -- https://www.reddit.com/r/golang/comments/qxghjv/plugin_already_loaded_when_a_plugin_is_loaded/ - - -## Setting up your environment - -It's important to understand the need for plugins to be compiled using exactly the same environment and build flags as the Gateway. To simplify this and minimise the risk of compatibility problems, we recommend the use of [Go workspaces](https://go.dev/blog/get-familiar-with-workspaces), to provide a consistent environment. - -To develop plugins without using the Tyk Plugin Compiler, you'll need: - -- Go (matching the version used in the Gateway, which you can determine using `go.mod`). -- Git to check out Tyk Gateway source code. -- A folder with the code that you want to build into plugins. - -We recommend that you set up a *Go workspace*, which, at the end, is going to contain: - -- `/tyk-release-x.y.z` - the Tyk Gateway source code -- `/plugins` - the plugins -- `/go.work` - the *Go workspace* file -- `/go.work.sum` - *Go workspace* package checksums - -Using the *Go workspace* ensures build compatibility between the plugins and Gateway. - -### 1. Checking out Tyk Gateway source code - -``` -git clone --branch release-5.3.6 https://github.com/TykTechnologies/tyk.git tyk-release-5.3.6 || true -``` - -This example uses a particular `release-5.3.6` branch, to match Tyk Gateway release 5.3.6. With newer `git` versions, you may pass `--branch v5.3.6` and it would use the tag. In case you want to use the tag it's also possible to navigate into the folder and issue `git checkout tags/v5.3.6`. - -### 2. Preparing the Go workspace - -Your Go workspace can be very simple: - -1. Create a `.go` file containing the code for your plugin. -2. Create a `go.mod` file for the plugin. -3. Ensure the correct Go version is in use. - -As an example, we can use the [CustomGoPlugin.go](https://github.com/TykTechnologies/custom-go-plugin/blob/master/go/src/CustomGoPlugin.go) sample as the source for our plugin as shown: - -``` -mkdir -p plugins -cd plugins -go mod init testplugin -go mod edit -go $(go mod edit -json go.mod | jq -r .Go) -wget -q https://raw.githubusercontent.com/TykTechnologies/custom-go-plugin/refs/heads/master/go/src/CustomGoPlugin.go -cd - -``` - -The following snippet provides you with a way to get the exact Go version used by Gateway from it's [go.mod](https://github.com/TykTechnologies/tyk/blob/release-5.3.6/go.mod#L3) file: - -- `go mod edit -json go.mod | jq -r .Go` (e.g. `1.22.7`) - -This should be used to ensure the version matches between gateway and the plugin. - -To summarize what was done: - -1. We created a plugins folder and initialzed a `go` project using `go mod` command. -2. Set the Go version of `go.mod` to match the one set in the Gateway. -3. Initialzied the project with sample plugin `go` code. - -At this point, we don't have a *Go workspace* but we will create one next so that we can effectively share the Gateway dependency across Go modules. - -### 3. Creating the Go workspace - -To set up the Go workspace, start in the directory that contains the Gateway and the Plugins folder. You'll first, create the `go.work` file to set up your Go workspace, and include the `tyk-release-5.3.6` and `plugins` folders. Then, navigate to the plugins folder to fetch the Gateway dependency at the exact commit hash and run `go mod tidy` to ensure dependencies are up to date. - -Follow these commands: - -``` -go work init ./tyk-release-5.3.6 -go work use ./plugins -commit_hash=$(cd tyk-release-5.3.6 && git rev-parse HEAD) -cd plugins && go get github.com/TykTechnologies/tyk@${commit_hash} && go mod tidy && cd - -``` - -The following snippet provides you to get the commit hash exactly, so it can be used with `go get`. - -- `git rev-parse HEAD` - -The Go workspace file (`go.work`) should look like this: - -``` -go 1.22.7 - -use ( - ./plugins - ./tyk-release-5.3.6 -) -``` - -### 4. Building and validating the plugin - -Now that your *Go workspace* is ready, you can build your plugin as follows: - -``` -cd tyk-release-5.3.6 && go build -tags=goplugin -trimpath . && cd - -cd plugins && go build -trimpath -buildmode=plugin . && cd - -``` - -These steps build both the Gateway and the plugin. - -You can use the Gateway binary that you just built to test that your new plugin loads into the Gateway without having to configure and then make a request to an API using this command: - -``` -./tyk-release-5.3.6/tyk plugin load -f plugins/testplugin.so -s AuthCheck -``` - -You should see an output similar to: - -``` -time="Oct 14 13:39:55" level=info msg="--- Go custom plugin init success! ---- " -[file=plugins/testplugin.so, symbol=AuthCheck] loaded ok, got 0x76e1aeb52140 -``` - -The log shows that the plugin has correctly loaded into the Gateway and that its `init` function has been successfully invoked. - -### 5. Summary - -In the preceding steps we have put together an end-to-end build environment for both the Gateway and the plugin. Bear in mind that runtime environments may have additional restrictions beyond Go version and build flags to which the plugin developer must pay attention. - -Compatibility in general is a big concern when working with Go plugins: as the plugins are tightly coupled to the Gateway, consideration must always be made for the build restrictions enforced by environment and configuration options. - -Continue with [Loading Go Plugins into Tyk](https://tyk.io/docs/product-stack/tyk-gateway/advanced-configurations/plugins/golang/loading-go-plugins/). - - -## Debugging Golang Plugins - -Plugins are native Go code compiled to a binary shared object file. The code may depend on `cgo` and require libraries like `libc` provided by the runtime environment. The following are some debugging steps for diagnosing issues arising from using plugins. - -### Warnings - -The [Plugin package - Warnings](https://pkg.go.dev/plugin#hdr-Warnings) section in the Go documentation outlines several requirements which can't be ignored when working with plugins. The most important restriction is the following: - -> Runtime crashes are likely to occur unless all parts of the program (the application and all its plugins) are compiled using exactly the same version of the toolchain, the same build tags, and the same values of certain flags and environment variables. - -### Using Incorrect Build Flags - -When working with Go plugins, it's easy to miss the restriction that the plugin at the very least must be built with the same Go version, and the same flags (notably `-trimpath`) as the Tyk Gateway on which it is to be used. - -If you miss an argument (for example `-trimpath`) when building the plugin, the Gateway will report an error when your API attempts to load the plugin, for example: - -``` -task: [test] cd tyk-release-5.3.6 && go build -tags=goplugin -trimpath . -task: [test] cd plugins && go build -buildmode=plugin . -task: [test] ./tyk-release-5.3.6/tyk plugin load -f plugins/testplugin.so -s AuthCheck -tyk: error: unexpected error: plugin.Open("plugins/testplugin"): plugin was built with a different version of package internal/goarch, try --help -``` - -Usually when the error hints at a standard library package, the build flags between the Gateway and plugin binaries don't match. - -Other error messages may be reported, depending on what triggered the issue. For example, if you omitted `-race` in the plugin but the gateway was built with `-race`, the following error will be reported: - -``` -plugin was built with a different version of package runtime/internal/sys, try --help -``` - -Strictly speaking: - -- Build flags like `-trimpath`, `-race` need to match. -- Go toolchain / build env needs to be exactly the same. -- For cross compilation you must use the same `CC` value for the build (CGO). -- `CGO_ENABLED=1`, `GOOS`, `GOARCH` must match with runtime. - -When something is off, you can check what is different by using the `go version -m` command for the Gateway (`go version -m tyk`) and plugin (`go version -m plugin.so`). Inspecting and comparing the output of `build` tokens usually yields the difference that caused the compatibility issue. - -### Plugin Compatibility Issues - -Below are some common situations where dependencies might cause issues: - -- The `Gateway` has a dependency without a `go.mod` file, but the plugin needs to use it. -- Both the `Gateway` and the plugin share a dependency. In this case, the plugin must use the exact same version as the `Gateway`. -- The plugin requires a different version of a shared dependency. - -Here’s how to handle each case: - -**Case 1: Gateway dependency lacks `go.mod`** - -- The plugin depends on the `Gateway`, which uses dependency *A*. -- *A* doesn’t have a `go.mod` file, so a pseudo version is generated during the build. -- Result: The build completes, but the plugin fails to load due to a version mismatch. - -**Solution:** Update the code to remove dependency *A*, or use a version of *A* that includes a `go.mod` file. - -**Case 2: Shared dependency with version matching** - -- The plugin and `Gateway` share a dependency, and this dependency includes a `go.mod` file. -- The version matches, and the dependency is promoted to *direct* in `go.mod`. -- Outcome: You’ll need to keep this dependency version in sync with the `Gateway`. - -**Case 3: Plugin requires a different version of a shared dependency** - -- The plugin and `Gateway` share a dependency, but the plugin needs a different version. -- If the other version is a major release (e.g., `/v4`), it’s treated as a separate package, allowing both versions to coexist. -- If it’s just a minor/patch difference, the plugin will likely fail to load due to a version conflict. - -**Recommendation:** For best results, use Go package versions that follow the Go module versioning (metaversion). However, keep in mind that many `Gateway` dependencies use basic `v1` semantic versioning, which doesn’t always enforce strict versioned import paths. - -### List plugin symbols - -Sometimes it's useful to list symbols from a plugin. For example, we can list the symbols as they are compiled into our testplugin: - -``` -# nm -gD testplugin.so | grep testplugin -00000000014db4b0 R go:link.pkghashbytes.testplugin -000000000170f7d0 D go:link.pkghash.testplugin -000000000130f5e0 T testplugin.AddFooBarHeader -000000000130f900 T testplugin.AddFooBarHeader.deferwrap1 -000000000130f980 T testplugin.AuthCheck -0000000001310100 T testplugin.AuthCheck.deferwrap1 -000000000130f540 T testplugin.init -0000000001310ce0 T testplugin.init.0 -0000000001ce9580 D testplugin..inittask -0000000001310480 T testplugin.InjectConfigData -0000000001310180 T testplugin.InjectMetadata -0000000001d2a3e0 B testplugin.logger -0000000001310cc0 T testplugin.main -0000000001310820 T testplugin.MakeOutboundCall -0000000001310c40 T testplugin.MakeOutboundCall.deferwrap1 -``` - -This command prints other symbols that are part of the binary. In the worst case, a build compatibility issue may cause a crash in the Gateway due to an unrecoverable error and this can be used to further debug the binaries produced. - -A very basic check to ensure Gateway/plugin compatibility is using the built in `go version -m `: - -``` -[output truncated] - build -buildmode=exe - build -compiler=gc - build -race=true - build -tags=goplugin - build -trimpath=true - build CGO_ENABLED=1 - build GOARCH=amd64 - build GOOS=linux - build GOAMD64=v1 - build vcs=git - build vcs.revision=1db1935d899296c91a55ba528e7b653aec02883b - build vcs.time=2024-09-24T12:54:26Z - build vcs.modified=false -``` - -These options should match between the Gateway binary and the plugin. You can use the command for both binaries and then compare the outputs. - diff --git a/tyk-docs/content/product-stack/tyk-gateway/advanced-configurations/plugins/golang/go-plugin-compiler.md b/tyk-docs/content/product-stack/tyk-gateway/advanced-configurations/plugins/golang/go-plugin-compiler.md deleted file mode 100644 index 85ea92bec4..0000000000 --- a/tyk-docs/content/product-stack/tyk-gateway/advanced-configurations/plugins/golang/go-plugin-compiler.md +++ /dev/null @@ -1,82 +0,0 @@ ---- -title: Plugin compiler -date: 2024-03-04 -description: "The Tyk Go Plugin compiler" -tags: ["custom plugin", "golang", "go plugin", "middleware", "plugin compiler", "compiler"] ---- - -Tyk provides a Plugin Compiler tool that will create a file that can be [loaded into Tyk]({{< ref "product-stack/tyk-gateway/advanced-configurations/plugins/golang/loading-go-plugins" >}}) to implement your desired custom logic. - -{{< note success >}} -**Note** - -The plugin compiler is not supported on Ubuntu 16.04 (Xenial Xerus) as it uses glibc 2.23 which is incompatible with our standard build environment. If you absolutely must have Go plugin support on Xenial, please contact Tyk support. - -{{< button_left href="https://tyk.io/contact/" color="green" content="Contact us" >}} -{{< /note >}} - -## Compiler options - -Most of the following arguments are applied only to developer flows. These aid development and testing purposes, and support of these varies across releases, due to changes in the Go ecosystem. - -The latest plugin compiler implements the following options: - -- `plugin_name`: output root file name (for example `plugin.so`) -- `build_id`: [optional] provides build uniqueness -- `GOOS`: [optional] override of GOOS (add `-e GOOS=linux`) -- `GOARCH`: [optional] override of GOARCH (add `-e GOARCH=amd64`) - -By default, if `build_id` is not provided, the gateway will not allow the plugin to be loaded twice. This is a restriction of the Go plugins standard library implementation. As long as the builds are made with a unique `build_id`, the same plugin can be loaded multiple times. - -When you provide a unique `build_id` argument, that also enables hot-reload compatibility of your `.so` plugin build, so that you would not need to restart the gateway, only reload it. - -- before 5.1: the plugin would be built in a filesystem path based on `build_id` -- since 5.2.4: the plugin compiler adjusts the Go module in use for the plugin. - -As the plugins are built with `-trimpath`, to omit local filesystem path details and improve plugin compatibility, the plugin compiler relies on the Go module itself to ensure each plugin build is unique. It modifies the plugin build `go.mod` file and imports to ensure a unique build. - -- [plugin package: Warnings](https://pkg.go.dev/plugin#hdr-Warnings) -- [golang#29525 - plugin: cannot open the same plugin with different names](https://github.com/golang/go/issues/29525) - -## Output filename - -Since v4.1.0 the plugin compiler has automatically added the following suffixes to the root filename provided in the `plugin_name` argument: - -- `{Gw-version}`: the Tyk Gateway version, for example, `v5.3.0` -- `{OS}`: the target operating system, for example `linux` -- `{arch}`: the target CPU architecture, for example, `arm64` - -Thus, if `plugin_name` is set to `plugin.so` then given these example values the output file will be: `plugin_v5.3.0_linux_arm64.so`. - -This enables you to have one directory with multiple versions of the same plugin targeting different Gateway versions. - -### Cross-compiling for different architectures and operating systems - -The Tyk Go Plugin Compiler can generate output for different architectures and operating systems from the one in which the compiler is run (cross-compiling). When you do this, the output filename will be suffixed with the target OS and architecture. - -You simply provide the target `GOOS` and `GOARCH` arguments to the plugin compiler, for example: - -```yaml -docker run --rm -v `pwd`:/plugin-source \ - --platform=linux/amd64 \ - tykio/tyk-plugin-compiler:v5.2.1 plugin.so $build_id linux arm64 -``` - -This command will cross-compile your plugin for a `linux/arm64` architecture. It will produce an output file named `plugin_v5.2.1_linux_arm64.so`. - -{{< note success >}} -**Note** - -If you are using the plugin compiler on MacOS, the docker run argument `--platform=linux/amd64` is necessary. The plugin compiler is a cross-build environment implemented with `linux/amd64`. -{{< /note >}} - -## Experimental options - -The plugin compiler also supports a set of environment variables being passed: - -- `DEBUG=1`: enables debug output from the plugin compiler process. -- `GO_TIDY=1`: runs go mod tidy to resolve possible dependency issues. -- `GO_GET=1`: invokes go get to retrieve the exact Tyk gateway dependency. - -These environment options are only available in the latest gateway and plugin compiler versions. -They are unsupported and are provided to aid development and testing workflows. diff --git a/tyk-docs/content/product-stack/tyk-gateway/advanced-configurations/plugins/golang/go-plugin-examples.md b/tyk-docs/content/product-stack/tyk-gateway/advanced-configurations/plugins/golang/go-plugin-examples.md deleted file mode 100644 index ceedf109e2..0000000000 --- a/tyk-docs/content/product-stack/tyk-gateway/advanced-configurations/plugins/golang/go-plugin-examples.md +++ /dev/null @@ -1,231 +0,0 @@ ---- -title: Example custom Go plugins -date: 2024-03-01 -description: "Example custom plugins in Golang" -tags: ["custom plugin", "golang", "go plugin", "middleware", "examples"] ---- - -This document provides a working example for providing specific functionality with a custom Go plugin. - -For more resources for writing plugins, please visit our [Plugin Hub]({{< ref "plugins/plugin-hub">}}). - -## Using a custom Go plugin as a virtual endpoint - -It is possible to send a response from the Golang plugin custom middleware. In the case that the HTTP response was sent: - -- The HTTP request processing is stopped and other middleware in the chain won't be used. -- The HTTP request round-trip to the upstream target won't happen -- Analytics records will still be created and sent to the analytics processing flow. - -Let's look at an example of how to send an HTTP response from the Tyk Golang plugin. Imagine that we need middleware which would send JSON with the current time if the request contains the parameter `get_time=1` in the request query string: - -```go -package main - -import ( - "encoding/json" - "net/http" - "time" -) - -func SendCurrentTime(rw http.ResponseWriter, r *http.Request) { - // check if we don't need to send reply - if r.URL.Query().Get("get_time") != "1" { - // allow request to be processed and sent to upstream - return - } - - //Prepare data to send - replyData := map[string]interface{}{ - "current_time": time.Now(), - } - - jsonData, err := json.Marshal(replyData) - if err != nil { - rw.WriteHeader(http.StatusInternalServerError) - return - } - - //Send HTTP response from the Golang plugin - rw.Header().Set("Content-Type", "application/json") - rw.WriteHeader(http.StatusOK) - rw.Write(jsonData) -} - -func main() {} -``` - -Let's build the plugin by running this command in the plugin project folder: - -```bash -go build -trimpath -buildmode=plugin -o /tmp/SendCurrentTime.so -``` - -Then let's edit the API spec to use this custom middleware: - -```json -"custom_middleware": { - "pre": [ - { - "name": "SendCurrentTime", - "path": "/tmp/SendCurrentTime.so" - } - ], - "post_key_auth": [], - "auth_check": {}, - "post": [], - "driver": "goplugin" -} -``` - -Let's check that we still perform a round trip to the upstream target if the request query string parameter `get_time` is not set: - -```bash -# curl http://localhost:8181/my_api_name/get -{ - "args": {}, - "headers": { - "Accept": "*/*", - "Accept-Encoding": "gzip", - "Host": "httpbin.org", - "User-Agent": "curl/7.54.0" - }, - "url": "https://httpbin.org/get" -} -``` - -Now let's check if our Golang plugin sends an HTTP 200 response (with JSON containing current time) when we set `get_time=1` query string parameter: - -```bash -# curl http://localhost:8181/my_api_name/get?get_time=1 -{"current_time":"2019-09-11T23:44:10.040878-04:00"} -``` - -Here we see that: - -- We've got an HTTP 200 response code. -- The response body has a JSON payload with the current time. -- The upstream target was not reached. Our Tyk Golang plugin served this request and stopped processing after the response was sent. - -## Performing custom authentication with a Golang plugin - -You can implement your own authentication method, using a Golang plugin and custom `"auth_check"` middleware. Ensure you set the two fields in Post Authentication Hook. - -Let's have a look at the code example. Imagine we need to implement a very trivial authentication method when only one key is supported (in the real world you would want to store your keys in some storage or have some more complex logic). - -```go -package main - -import ( - "net/http" - - "github.com/TykTechnologies/tyk/ctx" - "github.com/TykTechnologies/tyk/headers" - "github.com/TykTechnologies/tyk/user" -) - -func getSessionByKey(key string) *user.SessionState { - //Here goes our logic to check if the provided API key is valid and appropriate key session can be retrieved - - // perform auth (only one token "abc" is allowed) - if key != "abc" { - return nil - } - - // return session - return &user.SessionState{ - OrgID: "default", - Alias: "abc-session", - } -} - -func MyPluginAuthCheck(rw http.ResponseWriter, r *http.Request) { - //Try to get a session by API key - key := r.Header.Get(headers.Authorization) - session := getSessionByKey(key) - if session == nil { - // auth failed, reply with 403 - rw.WriteHeader(http.StatusForbidden) - return - } - - // auth was successful, add the session to the request's context so other middleware can use it - ctx.SetSession(r, session, true) - - // if compiling on a version older than 4.0.1, use this instead - // ctx.SetSession(r, session, key, true) -} - -func main() {} -``` - -A couple of notes about this code: - -- the package `"github.com/TykTechnologies/tyk/ctx"` is used to set a session in the request context - this is something `"auth_check"`-type custom middleware is responsible for. -- the package `"github.com/TykTechnologies/tyk/user"` is used to operate with Tyk's key session structure. -- our Golang plugin sends a 403 HTTP response if authentication fails. -- our Golang plugin just adds a session to the request context and returns if authentication was successful. - -Let's build the plugin by running the following command in the folder containing your plugin project: - -```bash -go build -trimpath -buildmode=plugin -o /tmp/MyPluginAuthCheck.so -``` - -Now let's check if our custom authentication works as expected (only one key `"abc"` should work). - -Authentication will fail with the wrong API key: - -```bash -# curl -v -H "Authorization: xyz" http://localhost:8181/my_api_name/get -* Trying ::1... -* TCP_NODELAY set -* Connected to localhost (::1) port 8181 (#0) -> GET /my_api_name/get HTTP/1.1 -> Host: localhost:8181 -> User-Agent: curl/7.54.0 -> Accept: */* -> Authorization: xyz -> -< HTTP/1.1 403 Forbidden -< Date: Wed, 11 Sep 2019 04:31:34 GMT -< Content-Length: 0 -< -* Connection #0 to host localhost left intact -``` - -Here we see that our custom middleware replied with a 403 response and request processing was stopped at this point. - -Authentication successful with the right API key: - -```bash -# curl -v -H "Authorization: abc" http://localhost:8181/my_api_name/get -* Trying ::1... -* TCP_NODELAY set -* Connected to localhost (::1) port 8181 (#0) -> GET /my_api_name/get HTTP/1.1 -> Host: localhost:8181 -> User-Agent: curl/7.54.0 -> Accept: */* -> Authorization: abc -> -< HTTP/1.1 200 OK -< Content-Type: application/json -< Date: Wed, 11 Sep 2019 04:31:39 GMT -< Content-Length: 257 -< -{ - "args": {}, - "headers": { - "Accept": "*/*", - "Accept-Encoding": "gzip", - "Authorization": "abc", - "Host": "httpbin.org", - "User-Agent": "curl/7.54.0" - }, - "url": "https://httpbin.org/get" -} -* Connection #0 to host localhost left intact -``` - -Here we see that our custom middleware successfully authenticated the request and we received a reply from the upstream target. \ No newline at end of file diff --git a/tyk-docs/content/product-stack/tyk-gateway/advanced-configurations/plugins/golang/loading-go-plugins.md b/tyk-docs/content/product-stack/tyk-gateway/advanced-configurations/plugins/golang/loading-go-plugins.md deleted file mode 100644 index 6d4f4638b5..0000000000 --- a/tyk-docs/content/product-stack/tyk-gateway/advanced-configurations/plugins/golang/loading-go-plugins.md +++ /dev/null @@ -1,114 +0,0 @@ ---- -title: Loading Custom Go Plugins into Tyk -date: 2024-03-01 -description: "How to load your custom Go plugins into Tyk" -tags: ["custom plugin", "golang", "go plugin", "middleware"] ---- - -For development purposes, we are going to load the plugin from local file storage. For production, you can use [bundles](#loading-a-tyk-golang-plugin-from-a-bundle) to deploy plugins to multiple gateways. - -In this example we are using a Tyk Classic API. In the API definition find the `custom_middleware` section and make it look similar to the snippet below. Tyk Dashboard users should use RAW API Editor to access this section. - -```json -"custom_middleware": { - "pre": [], - "post_key_auth": [], - "auth_check": {}, - "post": [ - { - "name": "AddFooBarHeader", - "path": "/plugin.so" - } - ], - "driver": "goplugin" -} -``` - -Here we have: - -- `driver` - Set this to `goplugin` (no value created for this plugin) which says to Tyk that this custom middleware is a Golang native plugin. -- `post` - This is the hook name. We use middleware with hook type `post` because we want this custom middleware to process the request right before it is passed to the upstream target (we will look at other types later). -- `post.name` - is your function name from the Go plugin project. -- `post.path` - is the full or relative (to the Tyk binary) path to the built plugin file (`.so`). Make sure Tyk has read access to this file. - -Also, let's set fields `"use_keyless": true` and `"target_url": "http://httpbin.org/"` - for testing purposes. We will test what request arrives to our upstream target and `httpbin.org` is a perfect fit for that. - -The API needs to be reloaded after that change (this happens automatically when you save the updated API in the Dashboard). - -Now your API with its Golang plugin is ready to process traffic: - -```bash -# curl http://localhost:8181/my_api_name/get -{ - "args": {}, - "headers": { - "Accept": "*/*", - "Accept-Encoding": "gzip", - "Foo": "Bar", - "Host": "httpbin.org", - "User-Agent": "curl/7.54.0" - }, - "url": "https://httpbin.org/get" -} -``` - -We see that the upstream target has received the header `"Foo": "Bar"` which was added by our custom middleware implemented as a native Golang plugin in Tyk. - -### Updating the plugin - -Loading an updated version of your plugin requires one of the following actions: - -- An API reload with a NEW path or file name of your `.so` file with the plugin. You will need to update the API spec section `"custom_middleware"`, specifying a new value for the `"path"` field of the plugin you need to reload. -- Tyk main process reload. This will force a reload of all Golang plugins for all APIs. - -If a plugin is loaded as a bundle and you need to update it you will need to update your API spec with a new `.zip` file name in the `"custom_middleware_bundle"` field. Make sure the new `.zip` file is uploaded and available via the bundle HTTP endpoint before you update your API spec. - -### Loading a Tyk Golang plugin from a bundle - -Currently we have loaded Golang plugins only directly from the file system. However, when you have multiple gateway instances, you need a more dynamic way to load plugins. Tyk offer bundle instrumentation [Plugin Bundles]({{< ref "plugins/how-to-serve-plugins/plugin-bundles" >}}). Using the bundle command creates an archive with your plugin, which you can deploy to the HTTP server (or AWS S3) and then your plugins will be fetched and loaded from that HTTP endpoint. - -You will need to set in `tyk.conf` these two fields: - -- `"enable_bundle_downloader": true` - enables the plugin bundles downloader -- `"bundle_base_url": "http://mybundles:8000/abc"` - specifies the base URL with the HTTP server where you place your bundles with Golang plugins (this endpoint must be reachable by the gateway) - -Also, you will need to specify the following field in your API spec: - -`"custom_middleware_bundle"` - here you place your filename with the bundle (`.zip` archive) to be fetched from the HTTP endpoint you specified in your `tyk.conf` parameter `"bundle_base_url"` - -To load a plugin, your API spec should set this field like so: - -```json -"custom_middleware_bundle": "FooBarBundle.zip" -``` - -Let's look at `FooBarBundle.zip` contents. It is just a ZIP archive with two files archived inside: - -- `AddFooBarHeader.so` - this is our Golang plugin -- `manifest.json` - this is a special file with meta information used by Tyk's bundle loader - -The contents of `manifest.json`: - -```yaml -{ - "file_list": [ - "AddFooBarHeader.so" - ], - "custom_middleware": { - "post": [ - { - "name": "AddFooBarHeader", - "path": "AddFooBarHeader.so" - } - ], - "driver": "goplugin" - }, - - ... -} -``` - -Here we see: - -- field `"custom_middleware"` with exactly the same structure we used to specify `"custom_middleware"` in API spec without bundle -- field `"path"` in section `"post"` now contains just a file name without any path. This field specifies `.so` filename placed in a ZIP archive with the bundle (remember how we specified `"custom_middleware_bundle": "FooBarBundle.zip"`). diff --git a/tyk-docs/content/product-stack/tyk-gateway/advanced-configurations/plugins/golang/writing-go-plugins.md b/tyk-docs/content/product-stack/tyk-gateway/advanced-configurations/plugins/golang/writing-go-plugins.md deleted file mode 100644 index f91fd3496b..0000000000 --- a/tyk-docs/content/product-stack/tyk-gateway/advanced-configurations/plugins/golang/writing-go-plugins.md +++ /dev/null @@ -1,280 +0,0 @@ ---- -title: Writing Custom Go Plugins -date: 2024-03-04 -description: "Features available when writing custom Go plugins" -tags: ["custom plugin", "golang", "go plugin", "middleware"] ---- - -Tyk's custom Go plugin middleware is very powerful as it provides you with access to different data types and functionality as explained in this section. - -Golang plugins are a very flexible and powerful way to extend the functionality of Tyk and uses the native Golang plugins API (see [go pkg/plugin docs](https://golang.org/pkg/plugin) for more details). - -Custom Go plugins can access various data objects relating to the API request: - -- [session]({{< ref "product-stack/tyk-gateway/advanced-configurations/plugins/golang/writing-go-plugins#accessing-the-session-object" >}}): the key session object provided by the client when making the API request -- [API definition]({{< ref "product-stack/tyk-gateway/advanced-configurations/plugins/golang/writing-go-plugins#accessing-the-api-definition" >}}): the Tyk OAS or Tyk Classic API definition for the requested API - -Custom Go plugins can also [terminate the request]({{< ref "product-stack/tyk-gateway/advanced-configurations/plugins/golang/writing-go-plugins#terminating-the-request" >}}) and stop further processing of the API request such that it is not sent to the upstream service. - -For more resources for writing plugins, please visit our [Plugin Hub]({{< ref "plugins/plugin-hub">}}). -To see an example of a Go plugin, please visit our [Go plugin examples]({{< ref "product-stack/tyk-gateway/advanced-configurations/plugins/golang/go-plugin-examples" >}}) page. - -## Accessing the internal state of a custom plugin - -A Golang plugin can be treated as a normal Golang package but: - -- the package name is always `"main"` and this package cannot be imported -- this package loads at run-time by Tyk and loads after all other Golang packages -- this package has to have an empty `func main() {}`. - -A Go plugin can have a declared `func init()` and it gets called only once (when Tyk loads this plugin for the first time for an API). - -It is possible to create structures or open connections to 3d party services/storage and then share them within every call and export the function in your Golang plugin. - -For example, here is an example of a Tyk Golang plugin with a simple hit counter: - -```go {linenos=true, linenostart=1} -package main - -import ( - "encoding/json" - "net/http" - "sync" - - "github.com/TykTechnologies/tyk/ctx" - "github.com/TykTechnologies/tyk/log" - "github.com/TykTechnologies/tyk/user" -) - -var logger = log.Get() - -// plugin exported functionality -func MyProcessRequest(rw http.ResponseWriter, r *http.Request) { - endPoint := r.Method + " " + r.URL.Path - logger.Info("Custom middleware, new hit:", endPoint) - - hitCounter := recordHit(endPoint) - logger.Debug("New hit counter value:", hitCounter) - - if hitCounter > 100 { - logger.Warning("Hit counter to high") - } - - reply := myReply{ - Session: ctx.GetSession(r), - Endpoint: endPoint, - HitCounter: hitCounter, - } - - jsonData, err := json.Marshal(reply) - if err != nil { - logger.Error(err.Error()) - rw.WriteHeader(http.StatusInternalServerError) - return - } - - rw.Header().Set("Content-Type", "application/json") - rw.WriteHeader(http.StatusOK) - rw.Write(jsonData) -} - -// called once plugin is loaded, this is where we put all initialisation work for plugin -// i.e. setting exported functions, setting up connection pool to storage and etc. -func init() { - hitCounter = make(map[string]uint64) -} - -// plugin internal state and implementation -var ( - hitCounter map[string]uint64 - hitCounterMu sync.Mutex -) - -func recordHit(endpoint string) uint64 { - hitCounterMu.Lock() - defer hitCounterMu.Unlock() - hitCounter[endpoint]++ - return hitCounter[endpoint] -} - -type myReply struct { - Session *user.SessionState `json:"session"` - Endpoint string `json:"endpoint"` - HitCounter uint64 `json:"hit_counter"` -} - -func main() {} -``` - -Here we see how the internal state of the Golang plugin is used by the exported function `MyProcessRequest` (the one we set in the API spec in the `"custom_middleware"` section). The map `hitCounter` is used to send internal state and count hits to different endpoints. Then our exported Golang plugin function sends an HTTP reply with endpoint hit statistics. - -## Accessing the API definition - -When Tyk passes a request to your plugin, the API definition is made available as part of the request context. - -{{< note success >}} -**Note** - -The API definition is accessed differently for Tyk OAS APIs and Tyk Classic APIs, as indicated in the following sections. If you use the wrong call for your API type, it will return `nil`. -{{< /note >}} - -### Working with Tyk OAS APIs - -The API definition can be accessed as follows: - -```go -package main - -import ( - "fmt" - "net/http" - - "github.com/TykTechnologies/tyk/ctx" -) - -func MyPluginFunction(w http.ResponseWriter, r *http.Request) { - oas := ctx.GetOASDefinition(r) - fmt.Println("OAS doc title is", oas.Info.Title) -} - -func main() {} -``` - -The invocation of `ctx.GetOASDefinition(r)` returns an `OAS` object containing the Tyk OAS API definition. -The Go data structure can be found [here](https://github.com/TykTechnologies/tyk/blob/master/apidef/oas/oas.go#L28). - -### Working with Tyk Classic APIs - -The API definition can be accessed as follows: - -```go -package main - -import ( - "fmt" - "net/http" - - "github.com/TykTechnologies/tyk/ctx" -) - -func MyPluginFunction(w http.ResponseWriter, r *http.Request) { - apidef := ctx.GetDefinition(r) - fmt.Println("API name is", apidef.Name) -} - -func main() {} -``` - -The invocation of `ctx.GetDefinition(r)` returns an APIDefinition object containing the Tyk Classic API Definition. -The Go data structure can be found [here](https://github.com/TykTechnologies/tyk/blob/master/apidef/api_definitions.go#L583). - -## Accessing the session object - -When Tyk passes a request to your plugin, the key session object is made available as part of the request context. This can be accessed as follows: - -```go -package main -import ( - "fmt" - "net/http" - "github.com/TykTechnologies/tyk/ctx" -) -func main() {} -func MyPluginFunction(w http.ResponseWriter, r *http.Request) { - session := ctx.GetSession(r) - fmt.Println("Developer ID:", session.MetaData["tyk_developer_id"] - fmt.Println("Developer Email:", session.MetaData["tyk_developer_email"] -} -``` - -The invocation of `ctx.GetSession(r)` returns an SessionState object. -The Go data structure can be found [here](https://github.com/TykTechnologies/tyk/blob/master/user/session.go#L106). - -Here is an [example](https://github.com/TykTechnologies/custom-plugin-examples/blob/master/plugins/go-auth-multiple_hook_example/main.go#L135) custom Go plugin that makes use of the session object. - -## Terminating the request - -You can terminate the request within your custom Go plugin and provide an HTTP response to the originating client, such that the plugin behaves similarly to a [virtual endpoint]({{< ref "advanced-configuration/compose-apis/virtual-endpoints" >}}). - -- the HTTP request processing is stopped and other middleware in the chain won't be used -- the HTTP request round-trip to the upstream target won't happen -- analytics records will still be created and sent to the analytics processing flow - -This [example]({{< ref "product-stack/tyk-gateway/advanced-configurations/plugins/golang/go-plugin-examples#using-a-custom-go-plugin-as-a-virtual-endpoint" >}}) demonstrates a custom Go plugin configured as a virtual endpoint. - -## Logging from a custom plugin - -Your plugin can write log entries to Tyk's logging system. - -To do so you just need to import the package `"github.com/TykTechnologies/tyk/log"` and use the exported public method `Get()`: - -```go {linenos=true, linenostart=1} -package main - -import ( - "net/http" - - "github.com/TykTechnologies/tyk/log" -) - -var logger = log.Get() - -// AddFooBarHeader adds custom "Foo: Bar" header to the request -func AddFooBarHeader(rw http.ResponseWriter, r *http.Request) { - logger.Info("Processing HTTP request in Golang plugin!!") - r.Header.Add("Foo", "Bar") -} - -func main() {} -``` - -### Monitoring instrumentation for custom plugins - -All custom middleware implemented as Golang plugins support Tyk's current built in instrumentation. - -The format for an event name with metadata is: `"GoPluginMiddleware:" + Path + ":" + SymbolName`, e.g., for our example, the event name will be: - -```text -"GoPluginMiddleware:/tmp/AddFooBarHeader.so:AddFooBarHeader" -``` - -The format for a metric with execution time (in nanoseconds) will have the same format but with the `.exec_time` suffix: - -```text -"GoPluginMiddleware:/tmp/AddFooBarHeader.so:AddFooBarHeader.exec_time" -``` - -## Creating a custom response plugin - -As explained [here]({{< ref "plugins/plugin-types/response-plugins" >}}), you can register a custom Go plugin to be triggered in the response middleware chain. You must configure the `driver` field to `goplugin` in the API definition when registering the plugin. - -### Response plugin method signature - -To write a response plugin in Go you need it to have a method signature as in the example below i.e. `func(http.ResponseWriter, *http.Response, *http.Request)`. -You can then access and modify any part of the request or response. User session and API definition data can be accessed as with other Go plugin hook types. - -```go -package main - -import ( - "bytes" - "encoding/json" - "io/ioutil" - "net/http" - -) - -// MyPluginResponse intercepts response from upstream -func MyPluginResponse(rw http.ResponseWriter, res *http.Response, req *http.Request) { - // add a header to our response object - res.Header.Add("X-Response-Added", "resp-added") - - // overwrite our response body - var buf bytes.Buffer - buf.Write([]byte(`{"message":"Hi! I'm a response plugin"}`)) - res.Body = ioutil.NopCloser(&buf) - -} - -func main() {} -``` diff --git a/tyk-docs/content/product-stack/tyk-gateway/middleware/endpoint-plugin-tyk-classic.md b/tyk-docs/content/product-stack/tyk-gateway/middleware/endpoint-plugin-tyk-classic.md deleted file mode 100644 index 0d7f3e78f7..0000000000 --- a/tyk-docs/content/product-stack/tyk-gateway/middleware/endpoint-plugin-tyk-classic.md +++ /dev/null @@ -1,73 +0,0 @@ ---- -title: Using the Per-Endpoint Plugin with Tyk Classic APIs -date: 2024-03-04 -description: "Using the per-endpoint custom plugin with Tyk Classic APIs" -tags: ["custom plugin", "golang", "go plugin", "middleware", "per-endpoint", "Tyk Classic", "Tyk Classic API"] ---- - -The [per-endpoint custom plugin]({{< ref "product-stack/tyk-gateway/middleware/endpoint-plugin" >}}) provides the facility to attach a custom Golang plugin at the end of the request processing chain. -This plugin allows you to add custom logic to the processing flow for the specific endpoint without adding to the processing complexity of other endpoints. -It can [terminate the request]({{< ref "product-stack/tyk-gateway/advanced-configurations/plugins/golang/writing-go-plugins#terminating-the-request" >}}), if required, -and hence can provide a [Virtual Endpoint]({{< ref "advanced-configuration/compose-apis/virtual-endpoints" >}}) style capability using the Go language, rather than JavaScript (as supported by the virtual endpoint middleware). - -This middleware is configured in the Tyk Classic API Definition. You can do this via the Tyk Dashboard API or in the API Designer. - -If you're using the newer Tyk OAS APIs, then check out the [Tyk OAS]({{< ref "product-stack/tyk-gateway/middleware/endpoint-plugin-tyk-oas" >}}) page. - -## Configuring the middleware in the Tyk Classic API Definition - -To enable the middleware you must add a new `go_plugin` object to the `extended_paths` section of your API definition. - -The `go_plugin` object has the following configuration: - -- `path`: the endpoint path -- `method`: the endpoint HTTP method -- `func_name`: this is the "symbol" or function name you are calling in your Go plugin once loaded - a function can be called by one or more APIs -- `plugin_path`: the relative path of the shared object containing the function you wish to call, one or many `.so` files can be called - -You can register multiple plugin functions for a single endpoint. Tyk will process them in the order they appear in the API definition. - -For example: -```json {linenos=true, linenostart=1} -{ - "extended_paths": { - "go_plugin": [ - { - "disabled": false, - "path": "/anything", - "method": "GET", - "plugin_path": "/middleware/myPlugin.so", - "func_name": "myUniqueFunctionName" - } - ] - } -} -``` - -In this example the per-endpoint custom plugin middleware has been configured for HTTP `GET` requests to the `/anything` endpoint. For any call made to this endpoint, Tyk will invoke the function `myUniqueFunctionName` in the file located at `/middleware/myPlugin.so`. - -## Configuring the middleware in the API Designer - -You can use the API Designer in the Tyk Dashboard to add the per-endpoint custom plugin middleware for your Tyk Classic API by following these steps. - -#### Step 1: Add an endpoint for the path and select the plugin - -From the **Endpoint Designer** add an endpoint that matches the path for which you want to trigger the custom plugin function. Select the **Go Plugin** plugin. - -{{< img src="/img/dashboard/endpoint-designer/endpointplugin.png" alt="Selecting the middleware" >}} - -#### Step 2: Locate the middleware in the raw API definition - -Once you have selected the middleware for the endpoint, you need to switch to the *Raw Definition* view and then locate the `go_plugin` section (you can search within the text editor window). - -{{< img src="/img/dashboard/endpoint-designer/endpointplugin_search.png" alt="Locating the middleware configuration" >}} - -#### Step 3: Configure the middleware - -Now you can directly edit the `plugin_path` and `func_name` to locate your compiled plugin function. - -{{< img src="/img/dashboard/endpoint-designer/endpointplugin_config.png" alt="Configuring the middleware" >}} - -#### Step 4: Save the API - -Use the *save* or *create* buttons to save the changes and activate the middleware. diff --git a/tyk-docs/content/product-stack/tyk-gateway/middleware/endpoint-plugin-tyk-oas.md b/tyk-docs/content/product-stack/tyk-gateway/middleware/endpoint-plugin-tyk-oas.md deleted file mode 100644 index d75568dae0..0000000000 --- a/tyk-docs/content/product-stack/tyk-gateway/middleware/endpoint-plugin-tyk-oas.md +++ /dev/null @@ -1,124 +0,0 @@ ---- -title: Using the Per-Endpoint Plugin with Tyk OAS APIs -date: 2024-03-04 -description: "Using the per-endpoint custom plugin with Tyk OAS APIs" -tags: ["custom plugin", "golang", "go plugin", "middleware", "per-endpoint", "Tyk OAS", "Tyk OAS API"] ---- - -The [per-endpoint custom plugin]({{< ref "product-stack/tyk-gateway/middleware/endpoint-plugin" >}}) provides the facility to attach a custom Go plugin at the end of the request processing chain. -This plugin allows you to add custom logic to the processing flow for the specific endpoint without adding to the processing complexity of other endpoints. -It can [terminate the request]({{< ref "product-stack/tyk-gateway/advanced-configurations/plugins/golang/writing-go-plugins#terminating-the-request" >}}) if required, -and provides a [Virtual Endpoint]({{< ref "advanced-configuration/compose-apis/virtual-endpoints" >}}) style capability using the Go language, rather than JavaScript (as supported by the virtual endpoint middleware). - -The middleware is configured in the [Tyk OAS API Definition]({{< ref "api-management/gateway-config-tyk-oas#operation" >}}). You can do this via the Tyk Dashboard API or in the API Designer. - -If you're using the legacy Tyk Classic APIs, then check out the [Tyk Classic]({{< ref "product-stack/tyk-gateway/middleware/endpoint-plugin-tyk-classic" >}}) page. - -## Configuring the middleware in the Tyk OAS API Definition - -The design of the Tyk OAS API Definition takes advantage of the `operationId` defined in the OpenAPI Document that declares both the path and method for which the middleware should be added. The `path` can contain wildcards in the form of any string bracketed by curly braces, for example `{user_id}`. These wildcards are so they are human readable and do not translate to variable names. Under the hood, a wildcard translates to the “match everything” regex of: `(.*)`. - -The endpoint plugin middleware (`postPlugins`) can be added to the `operations` section of the Tyk OAS Extension (`x-tyk-api-gateway`) in your Tyk OAS API Definition for the appropriate `operationId` (as configured in the `paths` section of your OpenAPI Document). - -The `postPlugins` object has the following configuration: - -- `enabled`: enable the middleware for the endpoint -- `functionName`: this is the name of the Go function that will be executed when the middleware is triggered -- `path`: the relative path to the source file containing the compiled Go code - -You can chain multiple plugin functions in an array. Tyk will process them in the order they appear in the API definition. - -For example: - -```json {hl_lines=["39-45"],linenos=true, linenostart=1} -{ - "components": {}, - "info": { - "title": "example-endpoint-plugin", - "version": "1.0.0" - }, - "openapi": "3.0.3", - "paths": { - "/anything": { - "get": { - "operationId": "anythingget", - "responses": { - "200": { - "description": "" - } - } - } - } - }, - "x-tyk-api-gateway": { - "info": { - "name": "example-endpoint-plugin", - "state": { - "active": true - } - }, - "upstream": { - "url": "http://httpbin.org/" - }, - "server": { - "listenPath": { - "value": "/example-endpoint-plugin/", - "strip": true - } - }, - "middleware": { - "operations": { - "anythingget": { - "postPlugins": [ - { - "enabled": true, - "functionName": "myUniqueFunctionName", - "path": "/middleware/myPlugin.so" - } - ] - } - } - } - } -} -``` - -In this example the per-endpoint custom plugin middleware has been configured for HTTP `GET` requests to the `/anything` endpoint. For any call made to this endpoint, Tyk will invoke the function `myUniqueFunctionName` in the file located at `/middleware/myPlugin.so`. - -The configuration above is a complete and valid Tyk OAS API Definition that you can import into Tyk to try out the per-endpoint custom plugin middleware. - -## Configuring the middleware in the API Designer - -Adding a per-endpoint custom plugin to your API endpoints is easy when using the API Designer in the Tyk Dashboard, simply follow these steps: - -#### Step 1: Add an endpoint - -From the **API Designer** add an endpoint that matches the path and method to which you want to apply the middleware. - -{{< img src="/img/dashboard/api-designer/tyk-oas-no-endpoints.png" alt="Tyk OAS API Designer showing no endpoints created" >}} - -{{< img src="/img/dashboard/api-designer/tyk-oas-add-endpoint.png" alt="Adding an endpoint to an API using the Tyk OAS API Designer" >}} - -{{< img src="/img/dashboard/api-designer/tyk-oas-no-middleware.png" alt="Tyk OAS API Designer showing no middleware enabled on endpoint" >}} - -#### Step 2: Select the Go Post-Plugin middleware - -Select **ADD MIDDLEWARE** and choose **Go Post-Plugin** from the *Add Middleware* screen. - -{{< img src="/img/dashboard/api-designer/tyk-oas-go-plugin.png" alt="Adding the Go Post-Plugin middleware" >}} - -#### Step 3: Configure the middleware - -You must provide the path to the compiled plugin and the name of the Go function that should be invoked by Tyk Gateway when the middleware is triggered. - -{{< img src="/img/dashboard/api-designer/tyk-oas-go-plugin-config.png" alt="Configuring the per-endpoint custom plugin" >}} - -#### Step 4: Save the API - -Select **ADD MIDDLEWARE** to save the middleware configuration. Remember to select **SAVE API** to apply the changes. - -{{< note success >}} -**Note** - -You are only able to add one custom plugin to each endpoint when using the API Designer, however you can add more by editing the API definition directly in the Raw Definition editor. -{{< /note >}} diff --git a/tyk-docs/content/product-stack/tyk-gateway/middleware/endpoint-plugin.md b/tyk-docs/content/product-stack/tyk-gateway/middleware/endpoint-plugin.md deleted file mode 100644 index 32b1ce02fa..0000000000 --- a/tyk-docs/content/product-stack/tyk-gateway/middleware/endpoint-plugin.md +++ /dev/null @@ -1,46 +0,0 @@ ---- -title: Per-Endpoint Custom Plugins -date: 2024-03-04 -description: "Detail of the per-endpoint custom plugin" -tags: ["custom plugin", "golang", "go plugin", "middleware", "per-endpoint"] ---- - -Tyk's custom plugin architecture allows you to deploy custom logic that will be invoked at certain points in the [middleware chain]({{< ref "concepts/middleware-execution-order" >}}) as Tyk processes requests to your APIs. - -At the API-level, there are several points in the processing flow where custom plugins can be "hooked", as explained [here]({{< ref "plugins/plugin-types/plugintypes" >}}). Each of these will be invoked for calls to any endpoint on an API. If you want to perform custom logic only for specific endpoints, you must include selective processing logic within the plugin. - -At the endpoint-level, Tyk provides the facility to attach a custom Golang plugin at the end of the request processing chain (immediately before the API-level post-plugin is executed). - -## When to use the per-endpoint custom plugin - -#### Aggregating data from multiple services - -From a custom plugin, you can make calls out to other internal and upstream APIs. You can then aggregate and process the responses, returning a single response object to the originating client. This allows you to configure a single externally facing API to simplify interaction with multiple internal services, leaving the heavy lifting to Tyk rather than standing up an aggregation service within your stack. - -#### Enforcing custom policies - -Tyk provides a very flexible middleware chain where you can combine functions to implement the access controls you require to protect your upstream services. Of course, not all scenarios can be covered by Tyk’s standard middleware functions, but you can use a custom plugin to apply whatever custom logic you require to optimize your API experience. - -#### Dynamic Routing - -With a custom plugin you can implement complex dynamic routing of requests made to a single external endpoint on to different upstream services. The flexibility of the virtual endpoint gives access to data within the request (including the key session) and also the ability to make calls to other APIs to make decisions on the routing of the request. It can operate as a super-powered URL rewrite middleware. - -## How the per-endpoint custom plugin works - -Tyk Gateway is written using Golang. This has a flexible plugin architecture which allows for custom code to be compiled separately from the gateway and then invoked natively by the gateway. When registering a custom Go plugin in the API definition, you must provide the location of the compiled plugin and also the name of the function to be invoked within that package. - -Go plugins must therefore be [compiled]({{< ref "product-stack/tyk-gateway/advanced-configurations/plugins/golang/go-plugin-compiler" >}}) and [loaded]({{< ref "product-stack/tyk-gateway/advanced-configurations/plugins/golang/loading-go-plugins" >}}) into the Gateway in order that the function named in the plugin configuration in the API definition can be located and executed at the appropriate stage in the request middleware processing chain. - -The custom code within the plugin has access to contextual data such as the session object and API definition. If required, it can [terminate the request]({{< ref "product-stack/tyk-gateway/advanced-configurations/plugins/golang/writing-go-plugins#terminating-the-request" >}}) and hence can provide a [Virtual Endpoint]({{< ref "advanced-configuration/compose-apis/virtual-endpoints" >}}) style capability using the Go language, rather than JavaScript (as supported by the virtual endpoint middleware). This can then act as a high-performance replacement for the JavaScript virtual endpoints or for cases when you want to make use of external libraries. - -
- -If you're using Tyk OAS APIs, then you can find details and examples of how to configure the per-endpoint custom plugin [here]({{< ref "product-stack/tyk-gateway/middleware/endpoint-plugin-tyk-oas" >}}). - -If you're using Tyk Classic APIs, then you can find details and examples of how to configure the per-endpoint custom plugin [here]({{< ref "product-stack/tyk-gateway/middleware/endpoint-plugin-tyk-classic" >}}). - - diff --git a/tyk-docs/content/product-stack/tyk-operator/advanced-configurations/custom-plugins.md b/tyk-docs/content/product-stack/tyk-operator/advanced-configurations/custom-plugins.md index 6a32c4d057..25cfc0a69a 100644 --- a/tyk-docs/content/product-stack/tyk-operator/advanced-configurations/custom-plugins.md +++ b/tyk-docs/content/product-stack/tyk-operator/advanced-configurations/custom-plugins.md @@ -15,11 +15,11 @@ Classic API to use custom plugins. ## Overview Using Tyk Classic APIs, developers can implement API-level custom plugins that can be optionally setup to execute for -each of the following [hooks]({{< ref "plugins/plugin-types/plugintypes#plugin-and-hook-types" >}}) in the API request -lifecycle: [Pre (Request)]({{< ref "plugins/plugin-types/request-plugins" >}}), [Authentication]({{< ref "plugins/plugin-types/auth-plugins/auth-plugins" >}}), -[Post (Request)]({{< ref "plugins/plugin-types/request-plugins" >}}), [Post -Authentication]({{< ref "plugins/plugin-types/request-plugins" >}}), [Response]({{< ref "plugins/plugin-types/response-plugins" >}}) -and [Analytics]({{< ref "plugins/plugin-types/analytics-plugins" >}}). Subsequently, users can execute, or “hook”, their +each of the following [hooks]({{< ref "api-management/plugins/plugin-types#plugin-and-hook-types" >}}) in the API request +lifecycle: [Pre (Request)]({{< ref "api-management/plugins/plugin-types#request-plugins" >}}), [Authentication]({{< ref "api-management/plugins/plugin-types#authentication-plugins" >}}), +[Post (Request)]({{< ref "api-management/plugins/plugin-types#request-plugins" >}}), [Post +Authentication]({{< ref "api-management/plugins/plugin-types#request-plugins" >}}), [Response]({{< ref "api-management/plugins/plugin-types#response-plugins" >}}) +and [Analytics]({{< ref "api-management/plugins/plugin-types#analytics-plugins" >}}). Subsequently, users can execute, or “hook”, their plugin into these phases of the API request lifecycle based on their specific use case. This document explains how to configure the following plugin types with different drivers (plugin languages): @@ -30,7 +30,7 @@ This document explains how to configure the following plugin types with differen - Post (Request) - Response -Please refer to [Analytics Plugins]({{< ref "plugins/plugin-types/analytics-plugins" >}}) to learn how to configure Analytics +Please refer to [Analytics Plugins]({{< ref "api-management/plugins/plugin-types#analytics-plugins" >}}) to learn how to configure Analytics plugins using Tyk Operator. --- @@ -115,7 +115,7 @@ complex processing workflows. For example, you might develop one plugin for logg request in the pre request phase. The `driver` configuration parameter describes the plugin implementation language. Please refer to the [supported -languages]({{< ref "/plugins/supported-languages#plugin-driver-names" >}}) section for list of supported plugin driver names. +languages]({{< ref "api-management/plugins/overview#plugin-driver-names" >}}) section for list of supported plugin driver names. Each plugin can have additional settings, such as: @@ -129,7 +129,7 @@ Each plugin can have additional settings, such as: At the endpoint-level, Tyk provides the facility to attach a custom Golang plugin at the end of the request processing chain (immediately before the API-level post-plugin is executed). Please note that -[per-endpoint]({{< ref "product-stack/tyk-gateway/middleware/endpoint-plugin" >}}) level plugins are not currently +[per-endpoint]({{< ref "api-management/plugins/plugin-types#per-endpoint-custom-plugins" >}}) level plugins are not currently supported by Tyk Operator. --- @@ -190,7 +190,7 @@ spec: At lines 14-18 we can see the _custom_middleware_ section contains the configuration for our plugin: - The `driver` configuration parameter is set to `otto` at line 15, since our plugin is a Javascript plugin. For other - valid values please refer to the [plugins driver page]({{< ref "plugins/supported-languages#plugin-driver-names" >}}). + valid values please refer to the [plugins driver page]({{< ref "api-management/plugins/overview#plugin-driver-names" >}}). - A plugin hook configuration block is specified at line 16, containing the `name` and `path` for our plugin. The plugin configuration block identifies the "hook" or phase in the API request lifecycle when Tyk Gateway will execute the plugin. In the example above the configuration block is for a `pre` request plugin that will be executed before any @@ -242,5 +242,5 @@ request lifecycle]({{< ref "concepts/middleware-execution-order" >}}). For a detailed guide, check out our blog post [How to Deploy Python Plugins in Tyk Running on Kubernetes](https://tyk.io/blog/how-to-deploy-python-plugins-in-tyk-running-on-kubernetes/), which walks you through all the steps required to create Python [plugin -bundles]({{< ref "plugins/how-to-serve-plugins/plugin-bundles" >}}), load them into the Tyk Gateway, and configure an API +bundles]({{< ref "api-management/plugins/overview#plugin-bundles" >}}), load them into the Tyk Gateway, and configure an API Definition to use them with the Tyk Operator. diff --git a/tyk-docs/content/shared/grpc-include.md b/tyk-docs/content/shared/grpc-include.md index f5189fed42..6a9cf731bc 100644 --- a/tyk-docs/content/shared/grpc-include.md +++ b/tyk-docs/content/shared/grpc-include.md @@ -1,7 +1,7 @@ --- --- -## Configure Tyk +**Configure Tyk** You will need to modify the Tyk global configuration file `tyk.conf` to use gRPC plugins. The following block should be present in this file: @@ -16,7 +16,7 @@ You will need to modify the Tyk global configuration file `tyk.conf` to use gRPC ``` -### tyk.conf Options +**tyk.conf Options** * `enable_coprocess`: This enables the plugin. * `coprocess_grpc_server`: This is the URL of our gRPC server. @@ -25,7 +25,7 @@ You will need to modify the Tyk global configuration file `tyk.conf` to use gRPC * `public_key_path`: Modify `public_key_path` in case you want to enforce the cryptographic check of the plugin bundle signatures. If the `public_key_path` isn't set, the verification process will be skipped and unsigned plugin bundles will be loaded normally. -### Configure an API Definition +**Configure an API Definition** There are two important parameters that we need to add or modify in the API definition. The first one is `custom_middleware_bundle` which must match the name of the plugin bundle file. If we keep this with the default name that the Tyk CLI tool uses, it will be `bundle.zip`: @@ -48,7 +48,7 @@ The second parameter is specific to this tutorial, and should be used in combina `enable_coprocess_auth` will instruct the Tyk gateway to authenticate this API using the associated custom authentication function that's implemented by our plugin. -### Configuration via the Tyk Dashboard +**Configuration via the Tyk Dashboard** To attach the plugin to an API, from the **Advanced Options** tab in the **API Designer** enter `bundle.zip` in the **Plugin Bundle ID** field. @@ -59,7 +59,7 @@ From the **Core Settings** tab in the **API Designer** select **Use Custom Authe {{< img src="/img/2.10/custom_auth_python.png" alt="Advanced Options" >}} -## Testing the Plugin +**Testing the Plugin** At this point we have our test HTTP server ready to serve the plugin bundle and the configuration with all the required parameters. diff --git a/tyk-docs/content/tyk-cloud.md b/tyk-docs/content/tyk-cloud.md index 58a174e449..af6f1c6a11 100644 --- a/tyk-docs/content/tyk-cloud.md +++ b/tyk-docs/content/tyk-cloud.md @@ -1781,8 +1781,8 @@ This section explains that you can use plugins with Tyk Cloud and links to detai Tyk Cloud allows you to take advantage of Tyk's plugin architecture that allows you to write powerful middleware. For this version of Tyk Cloud, we support the use of Python, JavaScript Middleware and Golang based plugins. For more details, see: -* [Python Plugins]({{< ref "plugins/supported-languages/rich-plugins/python/python" >}}) -* [JSVM]({{< ref "plugins/supported-languages/javascript-middleware" >}}) +* [Python Plugins]({{< ref "api-management/plugins/rich-plugins#overview" >}}) +* [JSVM]({{< ref "api-management/plugins/javascript#" >}}) * [Golang]({{< ref "#configure-plugins" >}}) Next you'll set up an Tyk Cloud Control Plane to use a Python Authentication Plugin. @@ -2090,14 +2090,14 @@ The manifest file contains information about your plugin file structure and how | File | Description | |-------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| custom_middleware | contains the middleware settings like the plugin driver you want to use (driver) and the hooks that your plugin will expose. We use the **auth_check** for this tutorial. For other hooks see [here]({{< ref "plugins/supported-languages/rich-plugins/rich-plugins-work#coprocess-dispatcher---hooks" >}}). | +| custom_middleware | contains the middleware settings like the plugin driver you want to use (driver) and the hooks that your plugin will expose. We use the **auth_check** for this tutorial. For other hooks see [here]({{< ref "api-management/plugins/rich-plugins#coprocess-dispatcher---hooks" >}}). | | file_list | contains the list of files to be included in the bundle. The CLI tool expects to find these files in the current working directory. | | name | references the name of the function that you implement in your plugin code: **MyAuthMiddleware** | | middleware.py | an additional file that contains the main implementation of our middleware. | **Step 3: Creating the middleware.py file** -* You import decorators from the Tyk module that gives us the Hook decorator, and we import [Tyk Python API helpers]({{< ref "plugins/supported-languages/rich-plugins/python/tyk-python-api-methods" >}}) +* You import decorators from the Tyk module that gives us the Hook decorator, and we import [Tyk Python API helpers]({{< ref "api-management/plugins/rich-plugins#tyk-python-api-methods" >}}) * You implement a middleware function and register it as a hook. The input includes the request object, the session object, the API meta data and its specification. The hook checks the authorization header for a specified value. In this tutorial we have called it `Authorization`. @@ -2147,7 +2147,7 @@ docker run \ -c '/opt/tyk-gateway/tyk bundle build -y' ``` -* A plugin bundle is a packaged version of the plugin, it may also contain a cryptographic signature of its contents. The -y flag tells the Tyk CLI tool to skip the signing process in order to simplify this tutorial. For more information on the Tyk CLI tool, see [here]({{< ref "plugins/how-to-serve-plugins/plugin-bundles#how-plugin-bundles-work" >}}). +* A plugin bundle is a packaged version of the plugin, it may also contain a cryptographic signature of its contents. The -y flag tells the Tyk CLI tool to skip the signing process in order to simplify this tutorial. For more information on the Tyk CLI tool, see [here]({{< ref "api-management/plugins/overview#how-plugin-bundles-work" >}}). * You should now have a `bundle.zip` file in the plugin working directory. * Next you will configure [uploading your plugin bundle file]({{< ref "#uploading-your-bundle" >}}) to your Amazon S3 bucket. diff --git a/tyk-docs/content/tyk-self-managed.md b/tyk-docs/content/tyk-self-managed.md index f2cc434e4a..0bd579b64c 100644 --- a/tyk-docs/content/tyk-self-managed.md +++ b/tyk-docs/content/tyk-self-managed.md @@ -1605,7 +1605,7 @@ Apps in private spaces don't enable SSL/TLS by default. It needs to be configure **Gateway Plugins** -In order to enable [rich plugins]({{< ref "plugins/supported-languages/rich-plugins" >}}) for the Gateway, please set the following Heroku config option to either `python` or `lua` depending on the type of plugins used: +In order to enable [rich plugins]({{< ref "api-management/plugins/rich-plugins#" >}}) for the Gateway, please set the following Heroku config option to either `python` or `lua` depending on the type of plugins used: ```{.copyWrapper} heroku config:set TYK_PLUGINS="python" -a infinite-plains-14949 ``` diff --git a/tyk-docs/data/alias.json b/tyk-docs/data/alias.json index 510bd61b01..f6fabbbe34 100644 --- a/tyk-docs/data/alias.json +++ b/tyk-docs/data/alias.json @@ -505,5 +505,62 @@ "advanced-configuration/integrate/3rd-party-identity-providers": "#", "tyk-stack/tyk-manager/sso/dashboard-login-okta-tib": "#oidc-with-okta", "tyk-stack/tyk-manager/sso/sso-auth0-tib": "#oidc-with-auth0", - "getting-started/key-concepts/openapi-specification": "#" + "getting-started/key-concepts/openapi-specification": "#", + "plugins": "#", + "plugins/get-started-selfmanaged/deploy-plugins": "#cicd---automating-your-plugin-builds", + "plugins/how-to-serve-plugins": "#plugin-deployment-types", + "plugins/how-to-serve-plugins/plugin-bundles": "#plugin-bundles", + "plugins/plugin-hub": "#plugins-hub", + "plugins/plugin-types/analytics-plugins": "#analytics-plugins", + "plugins/plugin-types/auth-plugins/auth-plugins": "#authentication-plugins", + "plugins/plugin-types/auth-plugins/id-extractor": "#plugin-caching-mechanism", + "plugins/plugin-types/plugintypes": "#plugin-types", + "plugins/plugin-types/request-plugins": "#request-plugins", + "plugins/plugin-types/response-plugins": "#response-plugins", + "plugins/supported-languages": "#supported-languages", + "plugins/supported-languages/golang": "#", + "plugins/supported-languages/javascript-middleware": "#", + "plugins/supported-languages/javascript-middleware/install-middleware/tyk-ce": "#installing-middleware-on-tyk-oss", + "plugins/supported-languages/javascript-middleware/install-middleware/tyk-hybrid": "#installing-middleware-on-tyk-hybrid", + "plugins/supported-languages/javascript-middleware/install-middleware/tyk-pro": "#installing-middleware-on-tyk-self-managed", + "plugins/supported-languages/javascript-middleware/javascript-api": "#javascript-api", + "plugins/supported-languages/javascript-middleware/middleware-scripting-guide": "#using-javascript-with-tyk", + "plugins/supported-languages/javascript-middleware/waf-js-plugin": "#waf-oss-modsecurity-plugin-example", + "plugins/supported-languages/rich-plugins": "#", + "plugins/supported-languages/rich-plugins/grpc": "#overview-1", + "plugins/supported-languages/rich-plugins/grpc/custom-auth-dot-net": "#create-custom-authentication-plugin-with-net", + "plugins/supported-languages/rich-plugins/grpc/custom-auth-nodejs": "#create-custom-authentication-plugin-with-net", + "plugins/supported-languages/rich-plugins/grpc/custom-auth-python": "#create-custom-authentication-plugin-with-python", + "plugins/supported-languages/rich-plugins/grpc/getting-started-python": "#getting-started-creating-a-python-grpc-server", + "plugins/supported-languages/rich-plugins/grpc/performance": "#performance", + "plugins/supported-languages/rich-plugins/grpc/request-transformation-java": "#create-a-request-transformation-plugin-with-java", + "plugins/supported-languages/rich-plugins/grpc/write-grpc-plugin": "#key-concepts", + "plugins/supported-languages/rich-plugins/luajit": "#using-lua", + "plugins/supported-languages/rich-plugins/luajit/requirements": "#requirements-5", + "plugins/supported-languages/rich-plugins/luajit/tutorial-add-demo-plugin-api": "#lua-plugin-tutorial", + "plugins/supported-languages/rich-plugins/python/custom-auth-python-tutorial": "#custom-authentication-plugin-tutorial", + "plugins/supported-languages/rich-plugins/python/performance": "#python-performance", + "plugins/supported-languages/rich-plugins/python/python": "#overview", + "plugins/supported-languages/rich-plugins/python/tutorial-add-demo-plugin-api": "#add-python-plugin-to-your-gateway", + "plugins/supported-languages/rich-plugins/python/tyk-python-api-methods": "#tyk-python-api-methods", + "plugins/supported-languages/rich-plugins/rich-plugins-data-structures": "#rich-plugins-data-structures", + "plugins/supported-languages/rich-plugins/rich-plugins-work": "#how-do-rich-plugins-work-", + "plugins/tutorials/quick-starts/go/dashboard": "#dashboard-plugins", + "plugins/tutorials/quick-starts/go/open-source": "#open-source-plugins", + "plugins/tutorials/quick-starts/go/quickstart": "#getting-started", + "product-stack/tyk-gateway/advanced-configurations/plugins/api-config/classic": "#tyk-classic-apis", + "product-stack/tyk-gateway/advanced-configurations/plugins/api-config/oas": "#tyk-oas-apis", + "product-stack/tyk-gateway/advanced-configurations/plugins/api-config/overview": "#api-configuration", + "product-stack/tyk-gateway/advanced-configurations/plugins/bundles/bundle-cli": "#bundler-cli-tool", + "product-stack/tyk-gateway/advanced-configurations/plugins/bundles/classic": "#tyk-classic-apis", + "product-stack/tyk-gateway/advanced-configurations/plugins/bundles/oas": "#tyk-oas-api-configuration", + "product-stack/tyk-gateway/advanced-configurations/plugins/golang/go-development-flow": "#custom-go-plugin-development-flow", + "product-stack/tyk-gateway/advanced-configurations/plugins/golang/go-plugin-compiler": "#plugin-compiler", + "product-stack/tyk-gateway/advanced-configurations/plugins/golang/go-plugin-examples": "#example-custom-go-plugins", + "product-stack/tyk-gateway/advanced-configurations/plugins/golang/loading-go-plugins": "#loading-custom-go-plugins-into-tyk", + "product-stack/tyk-gateway/advanced-configurations/plugins/golang/writing-go-plugins": "#writing-custom-go-plugins", + "product-stack/tyk-gateway/advanced-configurations/plugins/otel-plugins": "#instrumenting-plugins-with-opentelemetry", + "product-stack/tyk-gateway/middleware/endpoint-plugin-tyk-classic": "#using-the-per-endpoint-plugin-with-tyk-classic-apis", + "product-stack/tyk-gateway/middleware/endpoint-plugin-tyk-oas": "#using-the-per-endpoint-plugin-with-tyk-oas-apis", + "product-stack/tyk-gateway/middleware/endpoint-plugin": "#per-endpoint-custom-plugins" } diff --git a/tyk-docs/data/menu.yaml b/tyk-docs/data/menu.yaml index 57e0a442dd..5440d8389b 100644 --- a/tyk-docs/data/menu.yaml +++ b/tyk-docs/data/menu.yaml @@ -105,37 +105,22 @@ menu: path: /api-management/upstream-authentication category: Page show: False - - title: "Automations Tools" - category: Directory - show: True - menu: - - title: "Overview" - path: /api-management/automations - category: Page - show: True - - title: "API Management in Kubernetes" - path: /api-management/automations/operator - category: Page - show: True - - title: "Synchronizing Tyk Configuraiton" - path: /api-management/automations/sync - category: Page - show: True - - title: "Dashboard Configuration" - path: /api-management/dashboard-configuration + - title: "Rate Limiting" + path: /api-management/rate-limit category: Page show: True - - title: "Identity Management" - path: /api-management/external-service-integration + - title: "Security Policy and Access Keys" + path: /api-management/policies category: Page show: True - - title: "API Versioning" - path: /api-management/api-versioning + - title: "Certificates" + path: /api-management/certificates category: Page show: True - - title: "Troubleshooting" - path: /api-management/troubleshooting-debugging + - title: "Security Best Practices" category: Page + show: True + path: /api-management/security-best-practices - title: "Gateway Configuration" category: Directory show: True @@ -152,30 +137,74 @@ menu: path: /api-management/gateway-config-tyk-classic category: Page show: True - - title: "Rate Limiting" - path: /api-management/rate-limit - category: Page - show: True - - title: "Security Policy and Access Keys" - path: /api-management/policies + - title: "API Versioning" + path: /api-management/api-versioning category: Page show: True - - title: "Certificates" - path: /api-management/certificates + - title: "Identity Management" + path: /api-management/external-service-integration category: Page show: True - - title: "API Versioning" - path: /api-management/api-versioning - category: Page + - title: "Custom Plugins" + category: Directory show: True + menu: + - title: "Overview" + path: /api-management/plugins/overview + category: Page + show: True + - title: "Plugin Types" + path: /api-management/plugins/plugin-types + category: Page + show: True + - title: "Golang Plugins" + path: /api-management/plugins/golang + category: Page + show: True + - title: "Javscript Plugins" + path: /api-management/plugins/javascript + category: Page + show: True + - title: "Rich Plugins" + path: /api-management/plugins/rich-plugins + category: Page + show: True + - title: "Advance Configuration" + path: /api-management/plugins/advance-config + category: Page + show: True - title: "User Management" path: /api-management/user-management category: Page show: True + - title: "Automations Tools" + category: Directory + show: True + menu: + - title: "Overview" + path: /api-management/automations + category: Page + show: True + - title: "API Management in Kubernetes" + path: /api-management/automations/operator + category: Page + show: True + - title: "Synchronizing Tyk Configuraiton" + path: /api-management/automations/sync + category: Page + show: True + - title: "Dashboard Configuration" + path: /api-management/dashboard-configuration + category: Page + show: True - title: "Manage Environments" path: /api-management/multiple-environments category: Page show: True + - title: "Troubleshooting" + path: /api-management/troubleshooting-debugging + category: Page + show: True - title: "Manage APIs" category: Directory show: True @@ -544,14 +573,6 @@ menu: category: Directory show: False menu: - - title: "Security Best Practices" - category: Page - show: True - path: /api-management/security-best-practices - - title: "Troubleshooting" - path: /api-management/troubleshooting-debugging - category: Page - show: True - title: "Product Stack" path: /tyk-stack category: Label @@ -1105,291 +1126,6 @@ menu: path: /product-stack/tyk-gateway/advanced-configurations/distributed-tracing/open-tracing/zipkin category: Page show: True - - title: "Custom plugins" - category: Directory - show: True - menu: - - title: "Overview" - path: /plugins - category: Page - show: True - - title: "Quickstarts" - category: Directory - show: true - menu: - - title: "Go" - category: Directory - show: True - menu: - - title: "Overview" - path: /plugins/tutorials/quick-starts/go/quickstart - category: Page - show: True - - title: "Dashboard" - path: /plugins/tutorials/quick-starts/go/dashboard - category: Page - show: True - - title: "Open source" - path: /plugins/tutorials/quick-starts/go/open-source - category: Page - show: True - - title: "Plugins Hub" - path: /plugins/plugin-hub - catgory: Page - show: True - - title: "API Configuration" - category: Directory - show: true - menu: - - title: "Overview" - path: /product-stack/tyk-gateway/advanced-configurations/plugins/api-config/overview - category: Page - show: True - - title: "Tyk OAS configuration" - path: /product-stack/tyk-gateway/advanced-configurations/plugins/api-config/oas - category: Page - show: True - - title: "Tyk Classic configuration" - path: /product-stack/tyk-gateway/advanced-configurations/plugins/api-config/classic - category: Page - show: True - - title: "Plugin Types" - path: /plugins/plugin-types/plugintypes - category: Directory - show: True - menu: - - title: "Overview" - path: /plugins/plugin-types/plugintypes - category: Page - show: True - - title: "Request Plugins" - path: /plugins/plugin-types/request-plugins - category: Page - show: True - - title: "Authentication Plugins" - path: /plugins/plugin-types/auth-plugins/auth-plugins - category: Page - show: True - - title: "Authentication Plugin Caching" - path: /plugins/plugin-types/auth-plugins/id-extractor - category: Page - show: True - - title: "Response Plugins" - path: /plugins/plugin-types/response-plugins - category: Page - show: True - - title: "Analytics Plugins" - path: /plugins/plugin-types/analytics-plugins/ - category: Page - show: True - - title: "Per-endpoint Plugins" - category: Directory - show: True - menu: - - title: "Overview" - path: /product-stack/tyk-gateway/middleware/endpoint-plugin - category: Page - show: True - - title: "Tyk OAS configuration" - path: /product-stack/tyk-gateway/middleware/endpoint-plugin-tyk-oas - category: Page - show: True - - title: "Tyk Classic configuration" - path: /product-stack/tyk-gateway/middleware/endpoint-plugin-tyk-classic - category: Page - show: True - - title: "Plugin bundles" - category: Directory - show: True - menu: - - title: "Serving Plugins to Tyk Gateway" - path: /plugins/how-to-serve-plugins - category: Page - show: False - - title: "Overview" - path: /plugins/how-to-serve-plugins/plugin-bundles - category: Page - show: True - - title: "Tyk OAS configuration" - path: /product-stack/tyk-gateway/advanced-configurations/plugins/bundles/oas - category: Page - show: True - - title: "Tyk Classic configuration" - path: /product-stack/tyk-gateway/advanced-configurations/plugins/bundles/classic - category: Page - show: True - - title: "Bundle CLI tool" - path: /product-stack/tyk-gateway/advanced-configurations/plugins/bundles/bundle-cli - category: Page - show: True - - title: "Supported Languages" - category: Directory - show: True - menu: - - title: "Overview" - path: /plugins/supported-languages - category: Page - show: True - - title: "Golang" - category: Directory - show: True - menu: - - title: "Overview" - path: /plugins/supported-languages/golang - category: Page - show: True - - title: "Writing Go Plugins" - path: /product-stack/tyk-gateway/advanced-configurations/plugins/golang/writing-go-plugins - category: Page - show: True - - title: "Go Plugin Development Flow" - path: /product-stack/tyk-gateway/advanced-configurations/plugins/golang/go-development-flow - category: Page - show: True - - title: "Go Plugin Compiler" - path: /product-stack/tyk-gateway/advanced-configurations/plugins/golang/go-plugin-compiler - category: Page - show: True - - title: "Loading Go Plugins" - path: /product-stack/tyk-gateway/advanced-configurations/plugins/golang/loading-go-plugins - category: Page - show: True - - title: "Go Plugin Examples" - path: /product-stack/tyk-gateway/advanced-configurations/plugins/golang/go-plugin-examples - category: Page - show: True - - title: "OpenTelemetry Instrumentation" - path: /product-stack/tyk-gateway/advanced-configurations/plugins/otel-plugins - category: Page - show: True - - title: "Javascript" - category: Directory - show: True - menu: - - title: "Overview" - path: /plugins/supported-languages/javascript-middleware - category: Page - show: True - - title: "JS Scripting Guide" - path: /plugins/supported-languages/javascript-middleware/middleware-scripting-guide - category: Page - show: True - - title: "JavaScript API" - path: /plugins/supported-languages/javascript-middleware/javascript-api - category: Page - show: True - - title: "Using JS plugins with Tyk Self-Managed" - path: /plugins/supported-languages/javascript-middleware/install-middleware/tyk-pro - category: Page - show: True - - title: "Using JS plugins with Tyk Hybrid" - path: /plugins/supported-languages/javascript-middleware/install-middleware/tyk-hybrid - category: Page - show: True - - title: "Using JS plugins with Tyk OSS" - path: /plugins/supported-languages/javascript-middleware/install-middleware/tyk-ce - category: Page - show: True - - title: "WAF (OSS) ModSecurity Plugin example" - path: /plugins/supported-languages/javascript-middleware/waf-js-plugin - category: Page - show: True - - title: "Rich Plugins" - path: /plugins/supported-languages/rich-plugins - category: Page - show: True - - title: "Rich Plugins - How do They work?" - path: /plugins/supported-languages/rich-plugins/rich-plugins-work - category: Page - show: True - - title: "Rich Plugins - Data Structures" - path: /plugins/supported-languages/rich-plugins/rich-plugins-data-structures - category: Page - show: True - - title: "Rich Plugin - Python" - category: Directory - show: True - menu: - - title: "Overview" - path: /plugins/supported-languages/rich-plugins/python/python - category: Page - show: True - - title: "Tutorial- Custom Authentication Plugin" - path: /plugins/supported-languages/rich-plugins/python/custom-auth-python-tutorial - category: Page - show: True - - title: "Tutorial - Add Python Plugin To Your Gateway" - path: /plugins/supported-languages/rich-plugins/python/tutorial-add-demo-plugin-api - category: Page - show: True - - title: "Tyk Python API methods" - path: /plugins/supported-languages/rich-plugins/python/tyk-python-api-methods - category: Page - show: True - - title: "Performance" - path: /plugins/supported-languages/rich-plugins/python/performance - category: Page - show: True - - title: "Rich Plugin - gRPC" - category: Directory - show: True - menu: - - title: "Overview" - path: /plugins/supported-languages/rich-plugins/grpc - category: Page - show: True - - title: "Key Concepts" - path: /plugins/supported-languages/rich-plugins/grpc/write-grpc-plugin - category: Page - show: True - - title: "Getting Started" - path: /plugins/supported-languages/rich-plugins/grpc/getting-started-python - category: Page - show: True - - title: "Tutorials" - category: Directory - show: true - menu: - - title: "Create Java Request Transformation Plugin" - path: /plugins/supported-languages/rich-plugins/grpc/request-transformation-java - category: Page - show: True - - title: "Create .NET Custom Authentication Plugin" - path: /plugins/supported-languages/rich-plugins/grpc/custom-auth-dot-net - category: Page - show: True - - title: "Create NodeJS Custom Authentication Plugin" - path: /plugins/supported-languages/rich-plugins/grpc/custom-auth-nodejs - category: Page - show: True - - title: "Create Python Custom Authentication Plugin" - path: /plugins/supported-languages/rich-plugins/grpc/custom-auth-python - category: Page - show: True - - title: "Performance" - path: /plugins/supported-languages/rich-plugins/grpc/performance - category: Page - show: True - - title: "Rich Plugin - Lua" - category: Directory - show: True - menu: - - title: "LuaJIT" - path: /plugins/supported-languages/rich-plugins/luajit - category: Page - show: True - - title: "Requirements" - path: /plugins/supported-languages/rich-plugins/luajit/requirements - category: Page - show: True - - title: "Lua Plugin Tutorial" - path: /plugins/supported-languages/rich-plugins/luajit/tutorial-add-demo-plugin-api - category: Page - show: True - - title: "CICD Plugin Build" - path: /plugins/get-started-selfmanaged/deploy-plugins - category: Page - show: False - title: "Other (non-HTTP) protocols" category: Directory show: True From 05eb35d38e912c812d974f2dc6ef65e54e6aae73 Mon Sep 17 00:00:00 2001 From: Master Date: Tue, 11 Feb 2025 14:58:40 +0530 Subject: [PATCH 5/8] New IA - Portal Section (#5863) --- .../advanced-configuration/websockets.md | 2 +- .../content/api-management/api-versioning.md | 6 +- .../api-management/automations/operator.md | 4 +- .../api-management/dashboard-configuration.md | 12 +- .../external-service-integration.md | 8 +- .../gateway-config-tyk-classic.md | 6 +- .../api-management/gateway-config-tyk-oas.md | 12 +- .../tyk-oas-middleware.md | 2 +- .../api-management/multiple-environments.md | 4 +- .../api-management/plugins/javascript.md | 2 +- .../api-management/plugins/overview.md | 14 +- .../api-management/plugins/plugin-types.md | 15 +- .../api-management/plugins/rich-plugins.md | 4 +- tyk-docs/content/api-management/rate-limit.md | 10 +- .../api-management/security-best-practices.md | 16 +- .../troubleshooting-debugging.md | 2 +- .../reduce-latency/caching.md | 2 +- .../reduce-latency/caching/advanced-cache.md | 4 +- .../reduce-latency/caching/global-cache.md | 2 +- .../ext-oauth-middleware.md | 2 +- tyk-docs/content/developer-support/faq.md | 36 + .../release-notes/archived.md | 10 +- .../release-notes/dashboard.md | 30 +- .../release-notes/gateway.md | 40 +- .../release-notes/helm-chart.md | 4 +- .../developer-support/release-notes/portal.md | 80 +- .../developer-support/release-notes/tib.md | 2 +- tyk-docs/content/getting-started.md | 2 +- .../content/getting-started/installation.md | 2 +- .../key-concepts/graphql-subscriptions.md | 2 +- .../key-concepts/high-level-concepts.md | 2 +- .../import-an-oas-api.md | 4 +- .../versioning-an-oas-api.md | 2 +- tyk-docs/content/graphql.md | 4 +- tyk-docs/content/graphql/creating-gql-api.md | 14 +- tyk-docs/content/graphql/gql-headers.md | 2 +- .../content/graphql/graphql-playground.md | 2 +- .../content/graphql/graphql-websockets.md | 2 +- tyk-docs/content/graphql/introspection.md | 6 +- .../introspection/introspection-queries.md | 2 +- tyk-docs/content/graphql/persisted-queries.md | 2 +- tyk-docs/content/graphql/syncing-schema.md | 2 +- tyk-docs/content/key-concepts/grpc-proxy.md | 6 +- tyk-docs/content/portal/api-consumer.md | 380 ++ .../api-provider.md} | 347 +- tyk-docs/content/portal/customization.md | 3106 +++++++++++++++++ tyk-docs/content/portal/install.md | 992 ++++++ tyk-docs/content/portal/overview.md | 501 +++ .../enable-sso.md => portal/settings.md} | 134 +- .../tyk-charts/tyk-control-plane-chart.md | 4 +- .../tyk-charts/tyk-data-plane-chart.md | 2 +- .../product-stack/tyk-charts/tyk-oss-chart.md | 2 +- .../tyk-charts/tyk-stack-chart.md | 6 +- .../portal-1.13.0-list-of-endpoints.md | 2 +- .../deploy/bootstrapping-portal.md | 92 - .../deploy/configuration.md | 4 +- .../deploy/install-tyk-enterprise-portal.md | 79 - .../install-portal-using-docker-compose.md | 285 -- .../install-portal-using-docker.md | 294 -- .../install-portal-using-helm.md | 75 - .../install-portal-using-new-helm.md | 66 - .../install-portal-using-rpm.md | 118 - .../install-tyk-enterprise-portal/overview.md | 34 - .../create-api-product-and-plan.md | 134 - .../create-orgs-and-catalogs.md | 85 - .../customize-products-and-plans.md | 75 - .../getting-started/customize-sign-up-form.md | 69 - .../enterprise-portal-concepts.md | 101 - .../getting-started-with-enterprise-portal.md | 28 - .../setup-email-notifications.md | 66 - .../with-tyk-self-managed-as-provider.md | 62 - .../portal-customisation/.placeholder | 0 .../configure-webhooks.md | 374 -- .../customise-user-model.md | 80 - .../upgrading/theme-upgrades.md | 112 - .../advanced-configurations/synchroniser.md | 14 +- .../open-tracing/jaeger.md | 2 +- .../open-tracing/open-tracing-overview.md | 2 +- .../middleware/url-rewrite-middleware.md | 6 +- .../middleware/url-rewrite-tyk-classic.md | 6 +- .../middleware/url-rewrite-tyk-oas.md | 6 +- .../middleware/validate-request-tyk-oas.md | 2 +- .../advanced-configurations/custom-plugins.md | 2 +- .../setup-prometheus-pump.md | 4 +- .../common-configuration/batching.md | 6 +- .../configuration/inputs/broker.md | 4 +- .../configuration/inputs/http-client.md | 2 +- .../configuration/inputs/kafka.md | 2 +- .../configuration/inputs/overview.md | 4 +- .../configuration/outputs/broker.md | 2 +- .../configuration/outputs/http-client.md | 4 +- .../configuration/outputs/kafka.md | 4 +- .../configuration/outputs/overview.md | 2 +- .../configuration/processors/overview.md | 2 +- .../configuration/scanners/overview.md | 2 +- .../content/shared/api-def-authentication.md | 4 +- tyk-docs/content/shared/api-def-version.md | 4 +- tyk-docs/content/shared/create-api-include.md | 4 +- tyk-docs/content/shared/mdcb-config.md | 2 +- .../shared/oss-product-list-include.md | 2 +- .../transform-traffic/url-rewriting.md | 14 +- tyk-docs/content/tyk-cloud.md | 8 +- .../tyk-pump-configuration/datadog.md | 2 +- .../content/tyk-dashboard/configuration.md | 2 +- .../tyk-enterprise-developer-portal.md | 32 - .../api-consumer-portal.md | 21 - .../api-consumer-portal/access-api-product.md | 60 - .../api-consumer-portal/register-portal.md | 45 - .../api-consumer-portal/reset-password.md | 45 - .../tyk-portal-classic.md | 2 +- .../tyk-portal-classic/gluu-dcr.md | 2 +- .../tyk-portal-classic/keycloak-dcr.md | 2 +- .../portal-oauth-clients.md | 4 +- .../tyk-portal-classic/streams.md | 2 +- tyk-docs/content/tyk-multi-data-centre.md | 6 +- .../mdcb-configuration-options.md | 2 +- .../mdcb-example-minimising-latency.md | 4 +- .../setup-worker-data-centres.md | 2 +- tyk-docs/content/tyk-open-source.md | 8 +- .../content/tyk-oss-gateway/configuration.md | 2 +- tyk-docs/content/tyk-pump.md | 2 +- tyk-docs/content/tyk-pump/configuration.md | 4 +- .../tyk-pump-environment-variables.md | 2 +- tyk-docs/content/tyk-self-managed.md | 12 +- .../api-access/api-access.md | 29 - .../api-access/approve-requests.md | 42 - .../configuring-custom-rate-limit-keys.md | 82 - .../api-access/manage-apps-credentials.md | 36 - .../customise-enterprise-portal.md | 4 +- .../content-manager-workflow.md | 24 - .../create-new-page-template.md | 142 - .../full-customisation/developer-workflow.md | 25 - .../edit-manage-page-content.md | 44 - .../full-customisation/email-customization.md | 89 - .../file-structure-concepts.md | 189 - .../full-customisation/full-customisation.md | 19 - .../full-customisation/menus-customisation.md | 88 - .../full-customisation/templates.md | 1902 ---------- .../quick-customisation.md | 109 - .../create-api-product-and-plan.md | 67 - .../customise-menus.md | 60 - ...age-get-started-guides-for-api-products.md | 113 - .../publish-api-products-and-plans.md | 78 - .../with-tyk-self-managed-as-provider.md | 55 - .../managing-access/add-organisations.md | 46 - .../approve-self-registering-requests.md | 45 - .../managing-access/invite-codes.md | 48 - .../manage-api-consumer-organisations.md | 119 - .../managing-access/manage-api-consumers.md | 39 - .../managing-access/manage-api-users.md | 46 - .../managing-access/manage-catalogues.md | 59 - .../managing-access/managing-access.md | 23 - tyk-docs/content/universal-data-graph.md | 6 +- .../concepts/header_management.md | 4 +- .../concepts/reusing_response_fields.md | 2 +- .../datasources/graphql.md | 2 +- tyk-docs/data/alias.json | 55 +- tyk-docs/data/menu.yaml | 305 +- 158 files changed, 5788 insertions(+), 6683 deletions(-) create mode 100644 tyk-docs/content/portal/api-consumer.md rename tyk-docs/content/{tyk-stack/tyk-developer-portal/enterprise-developer-portal/api-access/dynamic-client-registration.md => portal/api-provider.md} (55%) create mode 100644 tyk-docs/content/portal/customization.md create mode 100644 tyk-docs/content/portal/install.md create mode 100644 tyk-docs/content/portal/overview.md rename tyk-docs/content/{tyk-stack/tyk-developer-portal/enterprise-developer-portal/managing-access/enable-sso.md => portal/settings.md} (86%) delete mode 100644 tyk-docs/content/product-stack/tyk-enterprise-developer-portal/deploy/bootstrapping-portal.md delete mode 100644 tyk-docs/content/product-stack/tyk-enterprise-developer-portal/deploy/install-tyk-enterprise-portal.md delete mode 100644 tyk-docs/content/product-stack/tyk-enterprise-developer-portal/deploy/install-tyk-enterprise-portal/install-portal-using-docker-compose.md delete mode 100644 tyk-docs/content/product-stack/tyk-enterprise-developer-portal/deploy/install-tyk-enterprise-portal/install-portal-using-docker.md delete mode 100644 tyk-docs/content/product-stack/tyk-enterprise-developer-portal/deploy/install-tyk-enterprise-portal/install-portal-using-helm.md delete mode 100644 tyk-docs/content/product-stack/tyk-enterprise-developer-portal/deploy/install-tyk-enterprise-portal/install-portal-using-new-helm.md delete mode 100644 tyk-docs/content/product-stack/tyk-enterprise-developer-portal/deploy/install-tyk-enterprise-portal/install-portal-using-rpm.md delete mode 100644 tyk-docs/content/product-stack/tyk-enterprise-developer-portal/deploy/install-tyk-enterprise-portal/overview.md delete mode 100644 tyk-docs/content/product-stack/tyk-enterprise-developer-portal/getting-started/create-api-product-and-plan.md delete mode 100644 tyk-docs/content/product-stack/tyk-enterprise-developer-portal/getting-started/create-orgs-and-catalogs.md delete mode 100644 tyk-docs/content/product-stack/tyk-enterprise-developer-portal/getting-started/customize-products-and-plans.md delete mode 100644 tyk-docs/content/product-stack/tyk-enterprise-developer-portal/getting-started/customize-sign-up-form.md delete mode 100644 tyk-docs/content/product-stack/tyk-enterprise-developer-portal/getting-started/enterprise-portal-concepts.md delete mode 100644 tyk-docs/content/product-stack/tyk-enterprise-developer-portal/getting-started/getting-started-with-enterprise-portal.md delete mode 100644 tyk-docs/content/product-stack/tyk-enterprise-developer-portal/getting-started/setup-email-notifications.md delete mode 100644 tyk-docs/content/product-stack/tyk-enterprise-developer-portal/getting-started/with-tyk-self-managed-as-provider.md delete mode 100644 tyk-docs/content/product-stack/tyk-enterprise-developer-portal/portal-customisation/.placeholder delete mode 100644 tyk-docs/content/product-stack/tyk-enterprise-developer-portal/portal-customisation/configure-webhooks.md delete mode 100644 tyk-docs/content/product-stack/tyk-enterprise-developer-portal/portal-customisation/customise-user-model.md delete mode 100644 tyk-docs/content/product-stack/tyk-enterprise-developer-portal/upgrading/theme-upgrades.md delete mode 100644 tyk-docs/content/tyk-developer-portal/tyk-enterprise-developer-portal.md delete mode 100644 tyk-docs/content/tyk-developer-portal/tyk-enterprise-developer-portal/api-consumer-portal.md delete mode 100644 tyk-docs/content/tyk-developer-portal/tyk-enterprise-developer-portal/api-consumer-portal/access-api-product.md delete mode 100644 tyk-docs/content/tyk-developer-portal/tyk-enterprise-developer-portal/api-consumer-portal/register-portal.md delete mode 100644 tyk-docs/content/tyk-developer-portal/tyk-enterprise-developer-portal/api-consumer-portal/reset-password.md delete mode 100644 tyk-docs/content/tyk-stack/tyk-developer-portal/enterprise-developer-portal/api-access/api-access.md delete mode 100644 tyk-docs/content/tyk-stack/tyk-developer-portal/enterprise-developer-portal/api-access/approve-requests.md delete mode 100644 tyk-docs/content/tyk-stack/tyk-developer-portal/enterprise-developer-portal/api-access/configuring-custom-rate-limit-keys.md delete mode 100644 tyk-docs/content/tyk-stack/tyk-developer-portal/enterprise-developer-portal/api-access/manage-apps-credentials.md delete mode 100644 tyk-docs/content/tyk-stack/tyk-developer-portal/enterprise-developer-portal/customise-enterprise-portal/full-customisation/content-manager-workflow.md delete mode 100644 tyk-docs/content/tyk-stack/tyk-developer-portal/enterprise-developer-portal/customise-enterprise-portal/full-customisation/create-new-page-template.md delete mode 100644 tyk-docs/content/tyk-stack/tyk-developer-portal/enterprise-developer-portal/customise-enterprise-portal/full-customisation/developer-workflow.md delete mode 100644 tyk-docs/content/tyk-stack/tyk-developer-portal/enterprise-developer-portal/customise-enterprise-portal/full-customisation/edit-manage-page-content.md delete mode 100644 tyk-docs/content/tyk-stack/tyk-developer-portal/enterprise-developer-portal/customise-enterprise-portal/full-customisation/email-customization.md delete mode 100644 tyk-docs/content/tyk-stack/tyk-developer-portal/enterprise-developer-portal/customise-enterprise-portal/full-customisation/file-structure-concepts.md delete mode 100644 tyk-docs/content/tyk-stack/tyk-developer-portal/enterprise-developer-portal/customise-enterprise-portal/full-customisation/full-customisation.md delete mode 100644 tyk-docs/content/tyk-stack/tyk-developer-portal/enterprise-developer-portal/customise-enterprise-portal/full-customisation/menus-customisation.md delete mode 100644 tyk-docs/content/tyk-stack/tyk-developer-portal/enterprise-developer-portal/customise-enterprise-portal/full-customisation/templates.md delete mode 100644 tyk-docs/content/tyk-stack/tyk-developer-portal/enterprise-developer-portal/customise-enterprise-portal/quick-customisation.md delete mode 100644 tyk-docs/content/tyk-stack/tyk-developer-portal/enterprise-developer-portal/getting-started-with-enterprise-portal/create-api-product-and-plan.md delete mode 100644 tyk-docs/content/tyk-stack/tyk-developer-portal/enterprise-developer-portal/getting-started-with-enterprise-portal/customise-menus.md delete mode 100644 tyk-docs/content/tyk-stack/tyk-developer-portal/enterprise-developer-portal/getting-started-with-enterprise-portal/manage-get-started-guides-for-api-products.md delete mode 100644 tyk-docs/content/tyk-stack/tyk-developer-portal/enterprise-developer-portal/getting-started-with-enterprise-portal/publish-api-products-and-plans.md delete mode 100644 tyk-docs/content/tyk-stack/tyk-developer-portal/enterprise-developer-portal/getting-started-with-enterprise-portal/with-tyk-self-managed-as-provider.md delete mode 100644 tyk-docs/content/tyk-stack/tyk-developer-portal/enterprise-developer-portal/managing-access/add-organisations.md delete mode 100644 tyk-docs/content/tyk-stack/tyk-developer-portal/enterprise-developer-portal/managing-access/approve-self-registering-requests.md delete mode 100644 tyk-docs/content/tyk-stack/tyk-developer-portal/enterprise-developer-portal/managing-access/invite-codes.md delete mode 100644 tyk-docs/content/tyk-stack/tyk-developer-portal/enterprise-developer-portal/managing-access/manage-api-consumer-organisations.md delete mode 100644 tyk-docs/content/tyk-stack/tyk-developer-portal/enterprise-developer-portal/managing-access/manage-api-consumers.md delete mode 100644 tyk-docs/content/tyk-stack/tyk-developer-portal/enterprise-developer-portal/managing-access/manage-api-users.md delete mode 100644 tyk-docs/content/tyk-stack/tyk-developer-portal/enterprise-developer-portal/managing-access/manage-catalogues.md delete mode 100644 tyk-docs/content/tyk-stack/tyk-developer-portal/enterprise-developer-portal/managing-access/managing-access.md diff --git a/tyk-docs/content/advanced-configuration/websockets.md b/tyk-docs/content/advanced-configuration/websockets.md index 691dfeaf6c..cce17d0071 100755 --- a/tyk-docs/content/advanced-configuration/websockets.md +++ b/tyk-docs/content/advanced-configuration/websockets.md @@ -54,4 +54,4 @@ An example Header configuration for using an Authentication Token with an API: {{< img src="/img/dashboard/system-management/websocket-auth-token.png" alt="Postman WebSocket Connection Result with Authorization token" >}} -See the [Access an API]({{< ref "/content/getting-started/create-api-key.md" >}}) tutorial for details on adding an Authentication Token to your APIs. \ No newline at end of file +See the [Access an API]({{< ref "getting-started/create-api-key" >}}) tutorial for details on adding an Authentication Token to your APIs. \ No newline at end of file diff --git a/tyk-docs/content/api-management/api-versioning.md b/tyk-docs/content/api-management/api-versioning.md index c394a68f66..98b3c13718 100644 --- a/tyk-docs/content/api-management/api-versioning.md +++ b/tyk-docs/content/api-management/api-versioning.md @@ -131,7 +131,7 @@ This is easy to do with Tyk. You can include the deprecated endpoint in the new Tyk's access control model supports very granular permissions to versioned APIs. -You can explicitly grant access to specific version(s) of an API by specifying only those version(s) in the [key]({{< ref "tyk-apis/tyk-gateway-api/token-session-object-details" >}}) (also known as an *authorization token*, *bearer token*, *access token*, *API token* or *token session object* - see [here]({{< ref "/api-management/client-authentication#use-auth-tokens" >}})). +You can explicitly grant access to specific version(s) of an API by specifying only those version(s) in the [key]({{< ref "tyk-apis/tyk-gateway-api/token-session-object-details" >}}) (also known as an *authorization token*, *bearer token*, *access token*, *API token* or *token session object* - see [here]({{< ref "api-management/client-authentication#use-auth-tokens" >}})).
{{< note success >}} @@ -158,7 +158,7 @@ If you're using the legacy Tyk Classic APIs, then check out the [Tyk Classic]({{ ### Controlling access to Tyk OAS API versions -You can explicitly grant access to specific version(s) of an API by specifying the individual API definitions for each version in the [key]({{< ref "tyk-apis/tyk-gateway-api/token-session-object-details" >}}) (also known as an *authorization token*, *bearer token*, *access token*, *API token* or *token session object* - see [here]({{< ref "/api-management/client-authentication#use-auth-tokens" >}})). +You can explicitly grant access to specific version(s) of an API by specifying the individual API definitions for each version in the [key]({{< ref "tyk-apis/tyk-gateway-api/token-session-object-details" >}}) (also known as an *authorization token*, *bearer token*, *access token*, *API token* or *token session object* - see [here]({{< ref "api-management/client-authentication#use-auth-tokens" >}})). When using Tyk OAS APIs there are some subtleties to the propagation of access control between versions of an API: - each version of an API is treated individually by Tyk Gateway, so access must be explicity granted for each version @@ -354,7 +354,7 @@ If you're using Tyk Operator then check out the [configuring API versioning in T ### Controlling access to Tyk Classic API versions -You can explicitly grant access to specific version(s) of an API by specifying only those version(s) in the [key]({{< ref "tyk-apis/tyk-gateway-api/token-session-object-details" >}}) (also known as an *authorization token*, *bearer token*, *access token*, *API token* or *token session object* - see [here]({{< ref "/api-management/client-authentication#use-auth-tokens" >}})). +You can explicitly grant access to specific version(s) of an API by specifying only those version(s) in the [key]({{< ref "tyk-apis/tyk-gateway-api/token-session-object-details" >}}) (also known as an *authorization token*, *bearer token*, *access token*, *API token* or *token session object* - see [here]({{< ref "api-management/client-authentication#use-auth-tokens" >}})). ### Configuring API versioning in the Tyk Classic API Definition diff --git a/tyk-docs/content/api-management/automations/operator.md b/tyk-docs/content/api-management/automations/operator.md index 5b673aa185..575328e564 100644 --- a/tyk-docs/content/api-management/automations/operator.md +++ b/tyk-docs/content/api-management/automations/operator.md @@ -255,8 +255,8 @@ To address this challenge, Tyk Operator allows you to directly reference certifi We assume you have already installed Tyk. If you don’t have it, check out [Tyk Cloud]({{}}) or [Tyk Self -Managed]({{}}) page. [Tyk Helm -Chart]({{}}) is the preferred (and easiest) way to install Tyk on Kubernetes. +Managed]({{}}) page. [Tyk Helm +Chart]({{}}) is the preferred (and easiest) way to install Tyk on Kubernetes. In order for policy ID matching to work correctly, Dashboard must have `allow_explicit_policy_id` and `enable_duplicate_slugs` set to `true` and Gateway must have `policies.allow_explicit_policy_id` set to `true`. diff --git a/tyk-docs/content/api-management/dashboard-configuration.md b/tyk-docs/content/api-management/dashboard-configuration.md index 0958f8b47b..8dec5d0fc1 100644 --- a/tyk-docs/content/api-management/dashboard-configuration.md +++ b/tyk-docs/content/api-management/dashboard-configuration.md @@ -118,19 +118,19 @@ The Dashboard exposes two APIs: To know more about Dashboard APIs, refer the following documents: - [Postman / Swagger / Open API specification]({{< ref "tyk-dashboard-api" >}}) - - [Dashboard API Usage Examples]({{< ref "" >}}) + - [Dashboard API Usage Examples]({{< ref "#dashboard-api-resources-and-usage" >}}) - **Dashboard Admin API**: Is used for system-level administration and initial setup tasks like managing organizations, initial user creation, backups/migrations and SSO setup. To know more about Dashboard Admin APIs, refer the following documents: - [Postman / Swagger / Open API specification]({{< ref "dashboard-admin-api" >}}) - - [Dashboard Admin API Usage Examples]({{< ref "" >}}) + - [Dashboard Admin API Usage Examples]({{< ref "#dashboard-admin-api-resources-and-usage" >}}) ### Authenticating with Dashboard APIs **Dashboard API** -The [Tyk Dashboard API]({{< ref "tyk-dashboard-api.md" >}}) is secured using an `Authorization` header that must be added to each request that is made. The **Tyk Dashboard API Access Credentials** `Authorization` key can be found within the Dashboard UI at the bottom of the **Edit User** section for a user. +The [Tyk Dashboard API]({{< ref "tyk-dashboard-api" >}}) is secured using an `Authorization` header that must be added to each request that is made. The **Tyk Dashboard API Access Credentials** `Authorization` key can be found within the Dashboard UI at the bottom of the **Edit User** section for a user. **Dashboard Admin API** @@ -3521,7 +3521,7 @@ The **Core Settings** tab provides access to configure basic settings for the AP - [Service Discovery]({{< ref "tyk-self-managed#service-discovery" >}}) - [API Ownership]({{< ref "api-management/user-management#api-ownership" >}}) - [API level rate limiting]({{< ref "api-management/rate-limit#configuring-the-rate-limiter-at-the-api-level" >}}) -- [Authentication]({{< ref "/api-management/client-authentication" >}}) +- [Authentication]({{< ref "api-management/client-authentication" >}}) ### Versions @@ -3988,7 +3988,7 @@ The use of the `#` qualifier to identify a category prevents the use of `#` in y {{< /note >}} ### Using API categories -API categories can be added and removed from APIs within the [API Designer]({{< ref "#api-designer" >}}), via the [Tyk Dashboard API]({{< ref "#tyk-dashboard-api" >}}), or via [Tyk Operator]({{< ref "/api-management/automations/operator#what-is-tyk-operator" >}}). +API categories can be added and removed from APIs within the [API Designer]({{< ref "#api-designer" >}}), via the [Tyk Dashboard API]({{< ref "#tyk-dashboard-api" >}}), or via [Tyk Operator]({{< ref "api-management/automations/operator#what-is-tyk-operator" >}}). #### API Designer The API Designer in the Tyk Dashboard UI provides a simple method for assigning APIs to categories, removing categories and filtering the API list by category. @@ -4037,7 +4037,7 @@ These endpoints will return information for categories across all APIs in the sy #### Tyk Operator -You can manage categories using Tyk Operator custom resources. Please refer to [Tyk Operator]({{}}) documentation to see how to manage API categories for Tyk OAS APIs and Tyk Classic APIs. +You can manage categories using Tyk Operator custom resources. Please refer to [Tyk Operator]({{}}) documentation to see how to manage API categories for Tyk OAS APIs and Tyk Classic APIs. ## Governance using API Templates diff --git a/tyk-docs/content/api-management/external-service-integration.md b/tyk-docs/content/api-management/external-service-integration.md index d530f790b8..56c9e319ee 100644 --- a/tyk-docs/content/api-management/external-service-integration.md +++ b/tyk-docs/content/api-management/external-service-integration.md @@ -499,10 +499,10 @@ There is a simplified flow, which does not require a corresponding OAuth client ## SSO with OpenID Connect (OIDC) -- Instruction on setting [SSO with Okta]({{< ref "" >}}) -- Instructions on setting [SSO with Auth0]({{< ref "" >}}) -- Instructions on setting [SSO with Keycloak]({{< ref "" >}}) -- Instructions on setting [SSO with AzureAD]({{< ref "" >}}) +- Instruction on setting [SSO with Okta]({{< ref "#oidc-with-okta" >}}) +- Instructions on setting [SSO with Auth0]({{< ref "#oidc-with-auth0" >}}) +- Instructions on setting [SSO with Keycloak]({{< ref "#oidc-with-keycloak" >}}) +- Instructions on setting [SSO with AzureAD]({{< ref "#oidc-with-azure-ad" >}}) ### OIDC with Azure AD diff --git a/tyk-docs/content/api-management/gateway-config-tyk-classic.md b/tyk-docs/content/api-management/gateway-config-tyk-classic.md index 890ca27273..56cb4f5cca 100644 --- a/tyk-docs/content/api-management/gateway-config-tyk-classic.md +++ b/tyk-docs/content/api-management/gateway-config-tyk-classic.md @@ -421,7 +421,7 @@ In this example we can see that the `Host` and `User-Agent` headers exist within * `jwt_signing_method`: Either HMAC or RSA - HMAC requires a shared secret while RSA requires a public key to use to verify against. Please see the section on JSON web tokens for more details on how to generate these. -* `jwt_source`: Must either be a base64 encoded valid RSA/HMAC key or a url to a resource serving JWK, this key will then be used to validate inbound JWT and throttle them according to the centralised JWT options and fields set in the configuration. See [Dynamic public key rotation using public JWKs URL]({{< ref "/api-management/client-authentication#use-json-web-tokens-jwt" >}}) for more details on JWKs. +* `jwt_source`: Must either be a base64 encoded valid RSA/HMAC key or a url to a resource serving JWK, this key will then be used to validate inbound JWT and throttle them according to the centralised JWT options and fields set in the configuration. See [Dynamic public key rotation using public JWKs URL]({{< ref "api-management/client-authentication#use-json-web-tokens-jwt" >}}) for more details on JWKs. * `jwt_identity_base_field`: Identifies the user or identity to be used in the Claims of the JWT. This will fallback to `sub` if not found. This field forms the basis of a new "virtual" token that gets used after validation. It means policy attributes are carried forward through Tyk for attribution purposes. @@ -641,7 +641,7 @@ For more details on CIDR notation, see [this Wikipedia entry](https://en.wikiped ### Tyk Operator -Please consult the Tyk Operator supporting documentation for an example of how to [configure an IP allow list]({{< ref "/api-management/automations/operator#ip-allowlist" >}}) with Tyk Operator. +Please consult the Tyk Operator supporting documentation for an example of how to [configure an IP allow list]({{< ref "api-management/automations/operator#ip-allowlist" >}}) with Tyk Operator. ## IP Blocklist (Middleware) @@ -662,4 +662,4 @@ For more details on CIDR notation, see [this Wikipedia entry](https://en.wikiped ### Tyk Operator -Please consult the Tyk Operator supporting documentation for an example of how to [configure an IP block list]({{< ref "/api-management/automations/operator#ip-blocklist" >}}) with Tyk Operator. +Please consult the Tyk Operator supporting documentation for an example of how to [configure an IP block list]({{< ref "api-management/automations/operator#ip-blocklist" >}}) with Tyk Operator. diff --git a/tyk-docs/content/api-management/gateway-config-tyk-oas.md b/tyk-docs/content/api-management/gateway-config-tyk-oas.md index cab0ec1a50..77b6070978 100644 --- a/tyk-docs/content/api-management/gateway-config-tyk-oas.md +++ b/tyk-docs/content/api-management/gateway-config-tyk-oas.md @@ -305,7 +305,7 @@ The basic idea here is that you can create a key based on a provided certificate #### Basic Authentication -Having the `http` type as the `securityScheme` defined in OAS API Definition, with the schema field set to basic, means that the *Tyk Gateway* uses basic authentication as the protection mechanism. It expects an access key in the same way as any other access method. For more information see the [Basic Authentication documentation]({{< ref "/api-management/client-authentication#use-basic-authentication" >}}). +Having the `http` type as the `securityScheme` defined in OAS API Definition, with the schema field set to basic, means that the *Tyk Gateway* uses basic authentication as the protection mechanism. It expects an access key in the same way as any other access method. For more information see the [Basic Authentication documentation]({{< ref "api-management/client-authentication#use-basic-authentication" >}}). Example: @@ -378,7 +378,7 @@ securitySchemes: { All you need to do in the Tyk configuration is to enable the authentication and specify the header details. -For more configuration options check the [JWT documentation]({{< ref "/api-management/client-authentication#use-json-web-tokens-jwt" >}}). +For more configuration options check the [JWT documentation]({{< ref "api-management/client-authentication#use-json-web-tokens-jwt" >}}). #### OAuth @@ -435,7 +435,7 @@ Example: } ``` -All you need to do in the Tyk configuration is to enable OAuth and specify the header details. See [OAuth documentation]({{< ref "/api-management/client-authentication#use-tyk-as-an-oauth-20-authorization-server" >}}) for more details. +All you need to do in the Tyk configuration is to enable OAuth and specify the header details. See [OAuth documentation]({{< ref "api-management/client-authentication#use-tyk-as-an-oauth-20-authorization-server" >}}) for more details. #### Multiple Authentication mechanisms @@ -502,7 +502,7 @@ For the above OAS configuration, Tyk looks at only the first `security` object: } } ``` -Please observe the presence of the `baseIdentityProvider` field, as this is required when enabling multiple authentication mechanisms at the same time. See [Multiple Auth documentation]({{< ref "/api-management/client-authentication#combine-authentication-methods" >}}) for more details. +Please observe the presence of the `baseIdentityProvider` field, as this is required when enabling multiple authentication mechanisms at the same time. See [Multiple Auth documentation]({{< ref "api-management/client-authentication#combine-authentication-methods" >}}) for more details. #### Other Authentication mechanisms @@ -512,7 +512,7 @@ For now, the only authentication mechanisms enabled with OAS API Definition conf - JSON Web Token (JWT) - Tyk as OAuth authorization server -To find out about the other client authentication methods supported by Tyk, see [Client Authentication]({{< ref "/api-management/client-authentication" >}}). +To find out about the other client authentication methods supported by Tyk, see [Client Authentication]({{< ref "api-management/client-authentication" >}}). #### Automatically protecting OAS API Definition APIs @@ -792,5 +792,5 @@ An API definition that combines an OpenAPI Description with the Tyk vendor field An API definition written in Tyk’s proprietary API Specification format. This fully describes how Tyk should be configured to resolve calls made to the API. An example of the structure of the Tyk Classic API definition is provided [here]({{< ref "api-management/gateway-config-tyk-classic" >}}). -{{< include "x-tyk-gateway.md" >}} +{{< include "x-tyk-gateway" >}} diff --git a/tyk-docs/content/api-management/manage-apis/tyk-oas-api-definition/tyk-oas-middleware.md b/tyk-docs/content/api-management/manage-apis/tyk-oas-api-definition/tyk-oas-middleware.md index 6c19e99fef..754f602695 100644 --- a/tyk-docs/content/api-management/manage-apis/tyk-oas-api-definition/tyk-oas-middleware.md +++ b/tyk-docs/content/api-management/manage-apis/tyk-oas-api-definition/tyk-oas-middleware.md @@ -27,7 +27,7 @@ Tyk Gateway's allow list function explicitly allows access just to paths that ar | Body | OAS API Definition | | Param | Path param: `{API-ID}` Query param: `allowList` | -For example, given the base Tyk OAS API used [here]({{< ref "/getting-started/using-oas-definitions/update-an-oas-api#tutorial-3-update-tyk-oas-api-definition-with-an-updated-openapi-definition" >}}) we can enable the `allowList` middleware with this `PATCH` command: +For example, given the base Tyk OAS API used [here]({{< ref "getting-started/using-oas-definitions/update-an-oas-api#tutorial-3-update-tyk-oas-api-definition-with-an-updated-openapi-definition" >}}) we can enable the `allowList` middleware with this `PATCH` command: ```curl curl --location --request PATCH 'http://{your-tyk-host}:{port}/tyk/apis/oas/{API-ID}?allowList=true' \ diff --git a/tyk-docs/content/api-management/multiple-environments.md b/tyk-docs/content/api-management/multiple-environments.md index 2e600e3bff..13ece7b3d3 100644 --- a/tyk-docs/content/api-management/multiple-environments.md +++ b/tyk-docs/content/api-management/multiple-environments.md @@ -119,7 +119,7 @@ The ID you use in with any Dashboard API integrations will change as the documen ### Use Tyk-Sync -You can also use our new Tyk-Sync tool which allows you to sync your APIs (and Policies) with a Version Control System (VCS). You can then move your APIs between environments. See [Tyk-Sync]({{< ref "/api-management/automations/sync" >}}) for more details. +You can also use our new Tyk-Sync tool which allows you to sync your APIs (and Policies) with a Version Control System (VCS). You can then move your APIs between environments. See [Tyk-Sync]({{< ref "api-management/automations/sync" >}}) for more details. ## Move Keys Between Environments @@ -275,7 +275,7 @@ As you can see, under the hood, the policy has been migrated correctly with targ ### Use Tyk-Sync -You can also use our new Tyk-Sync tool which allows you to sync your Policies (and APIs) with a Version Control System (VCS). You can then move your Policies between environments. See [Tyk-Sync]({{< ref "/api-management/automations/sync" >}}) for more details. +You can also use our new Tyk-Sync tool which allows you to sync your Policies (and APIs) with a Version Control System (VCS). You can then move your Policies between environments. See [Tyk-Sync]({{< ref "api-management/automations/sync" >}}) for more details. ## Gateway Sharding diff --git a/tyk-docs/content/api-management/plugins/javascript.md b/tyk-docs/content/api-management/plugins/javascript.md index 2ab5b2be14..b1227eb2cb 100644 --- a/tyk-docs/content/api-management/plugins/javascript.md +++ b/tyk-docs/content/api-management/plugins/javascript.md @@ -34,7 +34,7 @@ There are three middleware components that can be scripted with Tyk: 1. **Custom JavaScript plugins**: These execute either *pre* or *post* validation. A *pre* middleware component will execute before any session validation or token validation has taken place, while a *post* middleware component will execute after the request has been passed through all checks and is ready to be proxied upstream. -2. **Dynamic event handlers**: These components fire on certain API events (see the event handlers section), these are fired Async and do not have a cooldown timer. These are documented [here]({{< ref "/product-stack/tyk-gateway/basic-config-and-security/report-monitor-and-trigger-events/event-webhook-tyk-oas#set-up-a-webhook-event-handler-in-the-tyk-oas-api-definition" >}}). +2. **Dynamic event handlers**: These components fire on certain API events (see the event handlers section), these are fired Async and do not have a cooldown timer. These are documented [here]({{< ref "product-stack/tyk-gateway/basic-config-and-security/report-monitor-and-trigger-events/event-webhook-tyk-oas#set-up-a-webhook-event-handler-in-the-tyk-oas-api-definition" >}}). 3. **Virtual endpoints**: These are powerful programmable middleware invoked towards the end of the request processing chain. Unlike the custom JavaScript plugins, the virtual endpoint terminates the request. These are documented [here]({{< ref "advanced-configuration/compose-apis/virtual-endpoints" >}}). diff --git a/tyk-docs/content/api-management/plugins/overview.md b/tyk-docs/content/api-management/plugins/overview.md index 672892e368..578150561e 100644 --- a/tyk-docs/content/api-management/plugins/overview.md +++ b/tyk-docs/content/api-management/plugins/overview.md @@ -34,13 +34,13 @@ aliases: Plugins can be used to customize and enhance the capabilities of your APIs through integration with external services and databases to perform operations such as data transformation, custom authentication, logging and monitoring etc. -When Tyk receives an API request, it works through a [chain]({{< ref "middleware-execution-order" >}}) of processing *middleware* that is configured using the API definition. There are a large number of built-in middleware in the processing chain that are dedicated to performing [client authentication]({{< ref "/api-management/client-authentication" >}}), [request transformation]({{< ref "advanced-configuration/transform-traffic" >}}), [caching]({{< ref "basic-config-and-security/reduce-latency/caching" >}}) and many other processes before proxying the request to the upstream. +When Tyk receives an API request, it works through a [chain]({{< ref "middleware-execution-order" >}}) of processing *middleware* that is configured using the API definition. There are a large number of built-in middleware in the processing chain that are dedicated to performing [client authentication]({{< ref "api-management/client-authentication" >}}), [request transformation]({{< ref "advanced-configuration/transform-traffic" >}}), [caching]({{< ref "basic-config-and-security/reduce-latency/caching" >}}) and many other processes before proxying the request to the upstream. Tyk's custom plugin facility provides a powerful and flexible way to extend the middleware chain. It allows API developers to write custom middleware, in various programming languages, that can perform additional processing of requests and responses. For example, a custom authentication scheme can be implemented and executed on API requests, custom plugins can be used to provide integration with external services and databases, or additional processing can be performed on the response returned from the upstream. -There are several different stages of the [API request lifecycle]({{< ref "/concepts/middleware-execution-order" >}}) where custom plugins can be attached (or *hooked*) into the middleware chain allowing significant customization to meet your specific requirements. +There are several different stages of the [API request lifecycle]({{< ref "concepts/middleware-execution-order" >}}) where custom plugins can be attached (or *hooked*) into the middleware chain allowing significant customization to meet your specific requirements. Custom plugins are usually referred to by the location where they can be *hooked* into the middleware processing chain as follows: @@ -69,12 +69,12 @@ From the above illustration it can be seen that: Tyk supports four types of plugins: -1. **[Request Plugin]({{< ref "" >}})** -2. **[Authentication Plugin]({{< ref "" >}})** -3. **[Response Plugin]({{< ref "" >}})** -4. **[Analytics Plugin]({{< ref "" >}})** +1. **[Request Plugin]({{< ref "api-management/plugins/plugin-types#request-plugins" >}})** +2. **[Authentication Plugin]({{< ref "api-management/plugins/plugin-types#authentication-plugins" >}})** +3. **[Response Plugin]({{< ref "api-management/plugins/plugin-types#response-plugins" >}})** +4. **[Analytics Plugin]({{< ref "api-management/plugins/plugin-types#analytics-plugins" >}})** -To know more about plugin types and it's advanced configuration, refer the following [docs]({{< ref "" >}}). +To know more about plugin types and it's advanced configuration, refer the following [docs]({{< ref "api-management/plugins/plugin-types" >}}). ## Getting Started diff --git a/tyk-docs/content/api-management/plugins/plugin-types.md b/tyk-docs/content/api-management/plugins/plugin-types.md index 43b4429711..d98c96fc91 100644 --- a/tyk-docs/content/api-management/plugins/plugin-types.md +++ b/tyk-docs/content/api-management/plugins/plugin-types.md @@ -27,7 +27,7 @@ aliases: Custom Plugins enable users to execute custom code to complete tasks specific to their use case, allowing users to complete tasks that would not otherwise be possible using Tyk’s standard middleware options. -Tyk has a [pre-defined execution order]({{< ref "/concepts/middleware-execution-order" >}}) for the middleware which also includes **seven hooks** for the custom plugins. As such, users can execute, or `hook`, their plugin in these phases of the API request/response lifecycle based on their specific use case. +Tyk has a [pre-defined execution order]({{< ref "concepts/middleware-execution-order" >}}) for the middleware which also includes **seven hooks** for the custom plugins. As such, users can execute, or `hook`, their plugin in these phases of the API request/response lifecycle based on their specific use case. ## Plugin and Hook Types This table includes all the plugin types with the relevant hooks, their place in the execution chain, description and examples: @@ -51,10 +51,10 @@ There are two different options for the Post Plugin that is executed at t Tyk supports four types of plugins: -1. **[Request Plugin]({{< ref "" >}})** -2. **[Authentication Plugin]({{< ref "" >}})** -3. **[Response Plugin]({{< ref "" >}})** -4. **[Analytics Plugin]({{< ref "" >}})** +1. **[Request Plugin]({{< ref "#request-plugins" >}})** +2. **[Authentication Plugin]({{< ref "#authentication-plugins" >}})** +3. **[Response Plugin]({{< ref "#response-plugins" >}})** +4. **[Analytics Plugin]({{< ref "#analytics-plugins" >}})** ## Request Plugins @@ -330,9 +330,8 @@ spec: There are two advance configuratin with plugin types: -1. **[Per Endpoint Custom Plugin]({{< ref "" >}})** -2. **[Plugin Caching Mechanism for Authentication Plugin]({{< ref "" >}})** - +1. **[Per Endpoint Custom Plugin]({{< ref "#per-endpoint-custom-plugins" >}})** +2. **[Plugin Caching Mechanism for Authentication Plugin]({{< ref "#plugin-caching-mechanism" >}})** ## Per-Endpoint Custom Plugins diff --git a/tyk-docs/content/api-management/plugins/rich-plugins.md b/tyk-docs/content/api-management/plugins/rich-plugins.md index 891b1b0f5e..44c52c2801 100644 --- a/tyk-docs/content/api-management/plugins/rich-plugins.md +++ b/tyk-docs/content/api-management/plugins/rich-plugins.md @@ -501,7 +501,7 @@ For further details see [BasicAuthData](#basicauthdata). Added to sessions where a Tyk key (embedding a shared secret) is used as the public key for signing the JWT. The JWT token's KID header value references the ID of a Tyk key. See [JWTData](#jwtdata) for an example. `hmac_enabled` -When set to `true` this indicates generation of a [HMAC signature]({{< ref "/api-management/client-authentication#sign-requests-with-hmac" >}}) using the secret provided in `hmac_secret`. If the generated signature matches the signature provided in the *Authorization* header then authentication of the request has passed. +When set to `true` this indicates generation of a [HMAC signature]({{< ref "api-management/client-authentication#sign-requests-with-hmac" >}}) using the secret provided in `hmac_secret`. If the generated signature matches the signature provided in the *Authorization* header then authentication of the request has passed. `hmac_secret` The value of the HMAC shared secret. @@ -543,7 +543,7 @@ A UNIX timestamp that represents the time the session was last updated. Applicab This is a UNIX timestamp that signifies when a cached key or ID will expire. This relates to custom authentication, where authenticated keys can be cached to save repeated requests to the gRPC server. See [id_extractor]({{< ref "api-management/plugins/plugin-types#plugin-caching-mechanism" >}}) and [Auth Plugins]({{< ref "api-management/plugins/plugin-types#authentication-plugins" >}}) for additional information. `session_lifetime` -UNIX timestamp that denotes when the key will automatically expire. Any·subsequent API request made using the key will be rejected. Overrides the global session lifetime. See [Key Expiry and Deletion]({{< ref "/api-management/client-authentication#set-physical-key-expiry-and-deletion" >}}) for more information. +UNIX timestamp that denotes when the key will automatically expire. Any·subsequent API request made using the key will be rejected. Overrides the global session lifetime. See [Key Expiry and Deletion]({{< ref "api-management/client-authentication#set-physical-key-expiry-and-deletion" >}}) for more information. --- diff --git a/tyk-docs/content/api-management/rate-limit.md b/tyk-docs/content/api-management/rate-limit.md index d3b190ad35..bf70b3241d 100644 --- a/tyk-docs/content/api-management/rate-limit.md +++ b/tyk-docs/content/api-management/rate-limit.md @@ -235,13 +235,13 @@ effect for a minimum of the configured window duration (`per`). Gateway and Redi resource usage is increased with this option. This option can be enabled using the following configuration option -[enable_sentinel_rate_limiter]({{< ref "/tyk-oss-gateway/configuration.md#enable_sentinel_rate_limiter" >}}). +[enable_sentinel_rate_limiter]({{< ref "tyk-oss-gateway/configuration.md#enable_sentinel_rate_limiter" >}}). To optimize performance, you may configure your rate limits with shorter window duration values (`per`), as that will cause Redis to hold less data at any given moment. -Performance can be improved by enabling the [enable_non_transactional_rate_limiter]({{< ref "/tyk-oss-gateway/configuration.md#enable_non_transactional_rate_limiter" >}}). This leverages Redis Pipelining to enhance the performance of the Redis operations. Please consult the [Redis documentation](https://redis.io/docs/manual/pipelining/) for more information. +Performance can be improved by enabling the [enable_non_transactional_rate_limiter]({{< ref "tyk-oss-gateway/configuration.md#enable_non_transactional_rate_limiter" >}}). This leverages Redis Pipelining to enhance the performance of the Redis operations. Please consult the [Redis documentation](https://redis.io/docs/manual/pipelining/) for more information. Please consider the [Fixed Window Rate Limiter]({{< ref "#fixed-window-rate-limiter" >}}) algorithm as an alternative, if Redis performance is an issue. @@ -298,7 +298,7 @@ gateways, the DRL algorithm will be used if the rate limit exceeds 10 requests per second. If it is 10 or fewer, the system will fall back to the Redis Rate Limiter. -See [DRL Threshold]({{< ref "/tyk-oss-gateway/configuration.md#drl_threshold" >}}) for details on how to configure this feature. +See [DRL Threshold]({{< ref "tyk-oss-gateway/configuration.md#drl_threshold" >}}) for details on how to configure this feature. ## Rate Limiting Layers @@ -332,7 +332,7 @@ If you want to restrict an API client to a certain rate of requests to your APIs {{< note success >}} **Note** -It is assumed that the APIs being protected with a rate limit are using the [auth token]({{< ref "/api-management/client-authentication#use-auth-tokens" >}}) client authentication method and policies have already been created. +It is assumed that the APIs being protected with a rate limit are using the [auth token]({{< ref "api-management/client-authentication#use-auth-tokens" >}}) client authentication method and policies have already been created. {{< /note >}} You can configure this rate limit from the API Designer in Tyk Dashboard as follows: @@ -349,7 +349,7 @@ If you want to restrict API clients to a certain rate of requests for a specific {{< note success >}} **Note** -It is assumed that the APIs being protected with a rate limit are using the [auth token]({{< ref "/api-management/client-authentication#use-auth-tokens" >}}) client authentication method and policies have already been created. +It is assumed that the APIs being protected with a rate limit are using the [auth token]({{< ref "api-management/client-authentication#use-auth-tokens" >}}) client authentication method and policies have already been created. {{< /note >}} You can configure this rate limit from the API Designer in Tyk Dashboard as follows: diff --git a/tyk-docs/content/api-management/security-best-practices.md b/tyk-docs/content/api-management/security-best-practices.md index 1467aab8eb..4db8d8b8c3 100644 --- a/tyk-docs/content/api-management/security-best-practices.md +++ b/tyk-docs/content/api-management/security-best-practices.md @@ -42,8 +42,8 @@ It is the responsibility of the API to handle this form of attack since it can a Authentication is a vital aspect of API security. Failure to do so, as noted by OWASP, leads to *Broken Authentication* posing a significant risk to both API providers and data. Tyk provides the following features and authentication mechanisms: -- Prioritize secure methods, like [mutual TLS]({{< ref "/api-management/client-authentication#use-mutual-tls" >}}), over [basic authentication]({{< ref "/api-management/client-authentication#use-basic-authentication" >}}) wherever feasible. -- API owners can integrate external Identity Providers (IdPs) supporting methods like [OpenID Connect]({{< ref "/api-management/client-authentication#integrate-with-openid-connect-deprecated" >}}), [OAuth 2.0]({{< ref "/api-management/client-authentication#using-the-authorization-code-grant" >}}) or [JSON Web Tokens]({{< ref "/api-management/client-authentication#use-json-web-tokens-jwt" >}}). +- Prioritize secure methods, like [mutual TLS]({{< ref "api-management/client-authentication#use-mutual-tls" >}}), over [basic authentication]({{< ref "api-management/client-authentication#use-basic-authentication" >}}) wherever feasible. +- API owners can integrate external Identity Providers (IdPs) supporting methods like [OpenID Connect]({{< ref "api-management/client-authentication#integrate-with-openid-connect-deprecated" >}}), [OAuth 2.0]({{< ref "api-management/client-authentication#using-the-authorization-code-grant" >}}) or [JSON Web Tokens]({{< ref "api-management/client-authentication#use-json-web-tokens-jwt" >}}). - [Single Sign-On]({{< ref "api-management/external-service-integration#single-sign-on-sso" >}}) can be used for a centralized and trusted authentication source. API operators can choose from common authentication methods such as OAuth 2.0, LDAP, and SAML. - [Dynamic Client Registration]({{< ref "tyk-developer-portal/tyk-portal-classic/dynamic-client-registration#oauth-20-dynamic-client-registration-protocol-dcr" >}}), enables third-party authorization servers to issue client credentials via the Tyk Developer Portal. This streamlines Identity Management, eliminating the need to manage credentials across multiple systems. - Tyk's default authentication setup disallows credentials in URLs, reducing the risk of inadvertent exposure through backend logs. @@ -109,7 +109,7 @@ Tyk offers several mechanisms to help protect an API from Security Misconfigurat - Use [response header manipulation]({{< ref "advanced-configuration/transform-traffic/response-headers" >}}) to remove or modify API sensitive information. - Use [response body manipulation]({{< ref "advanced-configuration/transform-traffic/response-body" >}}) to remove or modify parts containing sensitive information. - [TLS]({{< ref "api-management/certificates" >}}) to ensure that clients use the right service and encrypt traffic. -- [Mutual TLS]({{< ref "/api-management/client-authentication#use-mutual-tls" >}}) with both the clients and API to ensure that callers with explicitly allowed client certificates can connect to the endpoints. +- [Mutual TLS]({{< ref "api-management/client-authentication#use-mutual-tls" >}}) with both the clients and API to ensure that callers with explicitly allowed client certificates can connect to the endpoints. - [Error Templates]({{< ref "advanced-configuration/error-templates" >}}) can be used to return a response body based on status code and content type. This can help minimize the implementation details returned to the client. - [CORS functionality]({{< ref "api-management/gateway-config-tyk-classic#cors" >}}) allows the Tyk Gateway to limit API access to particular browser-based consumers. - [Policy Path-Based Permissions]({{< ref "api-management/policies#secure-your-apis-by-method-and-path" >}}) and the [allowlist]({{< ref "product-stack/tyk-gateway/middleware/allow-list-tyk-oas#configuring-the-allow-list-in-the-tyk-oas-api-definition" >}}) plugin can be used to prevent clients from accessing API endpoints using non-authorized HTTP methods. For example, blocking the use of the DELETE method on an endpoint which should only accept GET requests. @@ -155,7 +155,7 @@ Authentication is the process of identifying API clients. It’s a broad topic, **Implement Appropriate Authentication** -Choose a suitable authentication approach based on the risk profile of the API. Is it publicly accessible or internal? Does it require user interaction or is it machine to machine? How sensitive is the data and functionality provided by the API? Simplistic approaches, such as [Bearer Tokens]({{< ref "/api-management/client-authentication#use-auth-tokens" >}}), can work for low risk, basic APIs, but for higher risk or more sophisticated APIs, it may be more appropriate to use a standards-based approach such as [OAuth 2.0]({{< ref "/api-management/client-authentication#use-tyk-as-an-oauth-20-authorization-server" >}}) or [OpenID Connect]({{< ref "/api-management/client-authentication#integrate-with-openid-connect-deprecated" >}}). Furthermore, using an [external identity provider]({{< ref "/api-management/client-authentication#integrate-with-external-authorization-server-deprecated" >}}) can deliver additional benefits, such as [single sign-on]({{< ref "api-management/external-service-integration#single-sign-on-sso" >}}), as well as multi-factor authentication approaches such as [biometric verification](https://www.okta.com/identity-101/biometrics-secure-authentication). +Choose a suitable authentication approach based on the risk profile of the API. Is it publicly accessible or internal? Does it require user interaction or is it machine to machine? How sensitive is the data and functionality provided by the API? Simplistic approaches, such as [Bearer Tokens]({{< ref "api-management/client-authentication#use-auth-tokens" >}}), can work for low risk, basic APIs, but for higher risk or more sophisticated APIs, it may be more appropriate to use a standards-based approach such as [OAuth 2.0]({{< ref "api-management/client-authentication#use-tyk-as-an-oauth-20-authorization-server" >}}) or [OpenID Connect]({{< ref "api-management/client-authentication#integrate-with-openid-connect-deprecated" >}}). Furthermore, using an [external identity provider]({{< ref "api-management/client-authentication#integrate-with-external-authorization-server-deprecated" >}}) can deliver additional benefits, such as [single sign-on]({{< ref "api-management/external-service-integration#single-sign-on-sso" >}}), as well as multi-factor authentication approaches such as [biometric verification](https://www.okta.com/identity-101/biometrics-secure-authentication). **Handle Data Securely** @@ -319,7 +319,7 @@ Modern APIs are often backed by large technology stacks composed of numerous com **Secure Connections** -Use [transport layer security]({{< ref "api-management/certificates" >}}) where possible. Most importantly, on inbound connections to the gateway and outbound connection from the gateway to the upstream API and other services. TLS can also be used as a form of authentication, using [Mutual TLS]({{< ref "/api-management/client-authentication#use-mutual-tls" >}}). +Use [transport layer security]({{< ref "api-management/certificates" >}}) where possible. Most importantly, on inbound connections to the gateway and outbound connection from the gateway to the upstream API and other services. TLS can also be used as a form of authentication, using [Mutual TLS]({{< ref "api-management/client-authentication#use-mutual-tls" >}}). **Limit Functionality** @@ -349,7 +349,7 @@ APIs need to be managed and governed just like any other resource, otherwise org **Restrict Version Availability**: Enforce the expiry of [API versions]({{< ref "api-management/api-versioning#tyk-classic-api-versioning-1" >}}) that are planned for deprecation, by setting a sunset date, beyond which they will not be accessible. -**Enforce Key Expiry**: In many situations it’s best to issue API keys that have a short, finite lifetime, especially when serving anonymous, external consumers. Set [expiry dates]({{< ref "api-management/policies#access-key-expiry" >}}) for API keys, or use ephemeral credentials with complementary authentication techniques that support key renewal, such as [OAuth 2.0 refresh tokens]({{< ref "/api-management/client-authentication#using-refresh-tokens" >}}) and [dynamic client registration]({{< ref "tyk-stack/tyk-developer-portal/enterprise-developer-portal/api-access/dynamic-client-registration" >}}). Then, should an API key fall into the wrong hands, there’s a chance that it has already expired. +**Enforce Key Expiry**: In many situations it’s best to issue API keys that have a short, finite lifetime, especially when serving anonymous, external consumers. Set [expiry dates]({{< ref "api-management/policies#access-key-expiry" >}}) for API keys, or use ephemeral credentials with complementary authentication techniques that support key renewal, such as [OAuth 2.0 refresh tokens]({{< ref "api-management/client-authentication#using-refresh-tokens" >}}) and [dynamic client registration]({{< ref "portal/api-provider#dynamic-client-registration" >}}). Then, should an API key fall into the wrong hands, there’s a chance that it has already expired. **Use Standardized Specifications**: Use the [OpenAPI Specification](https://en.wikipedia.org/wiki/OpenAPI_Specification) standard to design APIs. These specification documents act as a source of truth that can generate [API configuration]({{< ref "getting-started/using-oas-definitions/import-an-oas-api" >}}) and [portal documentation]({{< ref "tyk-apis/tyk-portal-api/portal-documentation#create-documentation" >}}). @@ -379,7 +379,7 @@ Tyk supports TLS connections and Mutual TLS. All TLS connections also support HT **Trusted Certificates** -As part of using Mutual TLS, you can create a list of [trusted certificates]({{< ref "/api-management/client-authentication#how-does-mutual-tls-work" >}}). +As part of using Mutual TLS, you can create a list of [trusted certificates]({{< ref "api-management/client-authentication#how-does-mutual-tls-work" >}}). **Certificate Pinning** @@ -397,7 +397,7 @@ Tyk supports various ways to secure your APIs, including: * OAuth 2.0 * OpenID Connect -See [Authentication and Authorization]({{< ref "/api-management/client-authentication" >}}) for more details. +See [Authentication and Authorization]({{< ref "api-management/client-authentication" >}}) for more details. **Security Policies** diff --git a/tyk-docs/content/api-management/troubleshooting-debugging.md b/tyk-docs/content/api-management/troubleshooting-debugging.md index 262abb8b50..e449d3ab0e 100644 --- a/tyk-docs/content/api-management/troubleshooting-debugging.md +++ b/tyk-docs/content/api-management/troubleshooting-debugging.md @@ -770,7 +770,7 @@ aliases: As AWS DocumentDB runs with TLS enabled, we require a way to run it without disabling the TLS verification. DocumentDB uses self-signed certs for verification, and provides a bundle with root certificates for this purpose, so we need a way to load this bundle. - Additionally DocumentDB can't be exposed to the local machine outside of the Amazon Virtual Private Cloud (VPC), which means that even if verification is turned on, it will always fail since if we use a SSH tunnel or a similar method, the domain will differ from the original. Also, it can have [Mutual TLS]({{< ref "/api-management/client-authentication#use-mutual-tls" >}}) enabled. + Additionally DocumentDB can't be exposed to the local machine outside of the Amazon Virtual Private Cloud (VPC), which means that even if verification is turned on, it will always fail since if we use a SSH tunnel or a similar method, the domain will differ from the original. Also, it can have [Mutual TLS]({{< ref "api-management/client-authentication#use-mutual-tls" >}}) enabled. So, in order to support it, we provide the following variables for both our [Tyk Analytics Dashboard]({{< ref "tyk-dashboard/configuration" >}}) and [Tyk Pump]({{< ref "tyk-pump/configuration" >}}): diff --git a/tyk-docs/content/basic-config-and-security/reduce-latency/caching.md b/tyk-docs/content/basic-config-and-security/reduce-latency/caching.md index 4c29d65a51..4f1f208bc4 100755 --- a/tyk-docs/content/basic-config-and-security/reduce-latency/caching.md +++ b/tyk-docs/content/basic-config-and-security/reduce-latency/caching.md @@ -62,7 +62,7 @@ Safe request caching at the API level is enabled by setting the `cache_all_safe_ This mode of operation is referred to as Global Caching because it is applied globally within the scope of a single API. Picking this approach will override any per-endpoint (per-path) caching configuration, so it’s not suitable if granular control is required. -Tyk does support safe request caching at the more granular, per-endpoint level, as described [here]({{< ref "/basic-config-and-security/reduce-latency/caching/advanced-cache#request-selective-cache-control">}}) - but `cache_all_safe_requests` must be set to `false` in that scenario. +Tyk does support safe request caching at the more granular, per-endpoint level, as described [here]({{< ref "basic-config-and-security/reduce-latency/caching/advanced-cache#request-selective-cache-control">}}) - but `cache_all_safe_requests` must be set to `false` in that scenario. #### Cache Timeout The cache timeout (Time-To-Live or TTL) value can be configured per API and is the maximum age for which Tyk will consider a cache entry to be valid. You should use this to optimize the tradeoff between reducing calls to your upstream service and potential for changes to the upstream data. diff --git a/tyk-docs/content/basic-config-and-security/reduce-latency/caching/advanced-cache.md b/tyk-docs/content/basic-config-and-security/reduce-latency/caching/advanced-cache.md index abd1c6a393..c0d52ad57d 100644 --- a/tyk-docs/content/basic-config-and-security/reduce-latency/caching/advanced-cache.md +++ b/tyk-docs/content/basic-config-and-security/reduce-latency/caching/advanced-cache.md @@ -5,7 +5,7 @@ tags: ["Caching", "Cache", "Endpoint Cache", "selective caching", "middleware", description: "Detail of the Endpoint Caching middleware" --- -On this page we describe how to configure Tyk's API response cache per endpoint within an API. This gives granular control over which paths are cached and allows you to vary cache configuration across API versions. For details on the API level (Global) cache you should refer to the [global-cache]({{< ref "/basic-config-and-security/reduce-latency/caching/global-cache">}}) configuration page. +On this page we describe how to configure Tyk's API response cache per endpoint within an API. This gives granular control over which paths are cached and allows you to vary cache configuration across API versions. For details on the API level (Global) cache you should refer to the [global-cache]({{< ref "basic-config-and-security/reduce-latency/caching/global-cache">}}) configuration page. When you use the API-level cache, Tyk will maintain a cache entry for each combination of request method, request path (endpoint) and API key (if authentication is enabled) for an API. The Endpoint Caching middleware gives you granular control over which paths are cached and allows you to vary cache configuration across API versions. @@ -28,7 +28,7 @@ For each endpoint in your API with endpoint caching middleware enabled, you can {{< note success >}} **Note** -It's important to note that the [cache all safe requests]({{< ref "/basic-config-and-security/reduce-latency/caching#global-cache-safe-requests">}}) feature of the API-level cache will overrule the per-endpoint configuration so you must ensure that both are not enabled for the same API. +It's important to note that the [cache all safe requests]({{< ref "basic-config-and-security/reduce-latency/caching#global-cache-safe-requests">}}) feature of the API-level cache will overrule the per-endpoint configuration so you must ensure that both are not enabled for the same API. {{< /note >}} #### Request-selective cache control diff --git a/tyk-docs/content/basic-config-and-security/reduce-latency/caching/global-cache.md b/tyk-docs/content/basic-config-and-security/reduce-latency/caching/global-cache.md index 203eaf5517..11472bea4c 100644 --- a/tyk-docs/content/basic-config-and-security/reduce-latency/caching/global-cache.md +++ b/tyk-docs/content/basic-config-and-security/reduce-latency/caching/global-cache.md @@ -9,7 +9,7 @@ menu: weight: 1 --- -_On this page we describe the use of Tyk's API response cache at the API level (Global); for details on the more advanced Endpoint level cache you should refer to [this]({{< ref "/basic-config-and-security/reduce-latency/caching/advanced-cache">}}) page._ +_On this page we describe the use of Tyk's API response cache at the API level (Global); for details on the more advanced Endpoint level cache you should refer to [this]({{< ref "basic-config-and-security/reduce-latency/caching/advanced-cache">}}) page._ Caching is configured separately for each API according to values you set within the API definition. Subsequently, the caching scope is restricted to an API definition, rather than being applied across the portfolio of APIs deployed in the Gateway. diff --git a/tyk-docs/content/basic-config-and-security/security/authentication-authorization/ext-oauth-middleware.md b/tyk-docs/content/basic-config-and-security/security/authentication-authorization/ext-oauth-middleware.md index 9d673a4a51..22020ef072 100644 --- a/tyk-docs/content/basic-config-and-security/security/authentication-authorization/ext-oauth-middleware.md +++ b/tyk-docs/content/basic-config-and-security/security/authentication-authorization/ext-oauth-middleware.md @@ -102,7 +102,7 @@ There could be cases when you don’t need to introspect a JWT access token from - a base64 encoded static secret - a valid JWK url in plain text - a valid JWK url in base64 encoded format -- `issuedAtValidationSkew` , `notBeforeValidationSkew`, `expiresAtValidationSkew` can be used to [configure clock skew]({{< ref "/api-management/client-authentication#use-json-web-tokens-jwt" >}}) for json web token validation. +- `issuedAtValidationSkew` , `notBeforeValidationSkew`, `expiresAtValidationSkew` can be used to [configure clock skew]({{< ref "api-management/client-authentication#use-json-web-tokens-jwt" >}}) for json web token validation. - `identityBaseField` - the identity key name for claims. If empty it will default to `sub`. ### Example: Tyk OAS API definition with JWT validation enabled diff --git a/tyk-docs/content/developer-support/faq.md b/tyk-docs/content/developer-support/faq.md index ca88c99d55..a3bd4ef6f6 100644 --- a/tyk-docs/content/developer-support/faq.md +++ b/tyk-docs/content/developer-support/faq.md @@ -100,6 +100,42 @@ This section lists commonly asked questions or frequently encountered issues and - For *Amazon RDS* users, check their [backup and restore documentation](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_CommonTasks.BackupRestore.html). To further enhance your PostgreSQL backup process, you can explore services like [AWS RDS Automated Backups](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_WorkingWithAutomatedBackups.html) if you're hosting your database on AWS. - For *CosmosDB* users check their [online backup and on-demand data restore documentation](https://learn.microsoft.com/en-us/azure/cosmos-db/postgresql/concepts-backup) +## Enterprise Developer Portal + +1. **What happens if the Portal goes down ?** + + In the event of the portal application being down, the other components of the Tyk Stack will remain unaffected. + This means your APIs will still be operational, and analytics will continue to be recorded. + Developers will also be able to use their credentials for both oAuth2.0 and API Keys APIs. + + However, since the portal application is down, developers won't be able to access their credentials or the analytical dashboard, request access to new API Products, or revoke or rotate their access credentials. + Additionally, admin users won't be able to use the portal, whether through its UI or APIs. + This means you won't be able to create, change, or remove any item managed by the portal, such as developers, organizations, content pages, API Products, plans, and more. + + Despite this, you still have some control over access credentials. + If you need to rotate or remove access credentials, you can do so directly in the Tyk Dashboard or in your identity provider. + +2. **What happens if the Dashboard goes down ?** + + If the Tyk Dashboard goes down, developers will still be able to access their access credentials, but they won't be able to rotate or remove their existing credentials, or request access to API Products. + Additionally, the API Analytics dashboard will be compromised. + + However, API traffic will remain unaffected, meaning that your APIs will continue to be operational, and analytics will continue to be recorded. + + In terms of admin functionality, the only limitation will be the inability to approve or reject access requests or revoke or rotate access credentials. + + +3. **Does the portal support SQL databases for storing the portal's CMS assets ?** + + {{< note success >}} + **Note** + + Tyk no longer supports SQLite as of Tyk 5.7.0. To avoid disruption, please transition to [PostgreSQL]({{< ref"tyk-self-managed#postgresql" >}}), [MongoDB]({{< ref "tyk-self-managed#mongodb" >}}), or one of the listed compatible alternatives. + {{< /note >}} + + The Enterprise Developer Portal supports SQL databases (MariaDB, MySQL, and PostgreSQL) for storing the portal's CMS assets. + During the bootstrap process, the portal will create the appropriate tables in the main database. The only thing required to enable SQL storage for the portal's assets is to specify the `db` [storage type]({{< ref "product-stack/tyk-enterprise-developer-portal/deploy/configuration#portal_storage" >}}) either via a config file or an environment variable. + ## Tyk Gateway 1. **How to Check Your Gateway Version ?** diff --git a/tyk-docs/content/developer-support/release-notes/archived.md b/tyk-docs/content/developer-support/release-notes/archived.md index a5cd1e4891..dc3e0d34ff 100644 --- a/tyk-docs/content/developer-support/release-notes/archived.md +++ b/tyk-docs/content/developer-support/release-notes/archived.md @@ -66,7 +66,7 @@ Other changes: Now you can set granular permissions on per user basis, by injecting permissions to the "scope" claim of a JSON Web Token. To make it work you need to provide mapping between the scope and policy ID, and thanks to enchanced policy merging capabilities mentioned above, Tyk will read the scope value from the JWT and will generate dynamic access rules. Your JWT scopes can look like `"users:read companies:write"` or similar, it is up to your imagination. OpenID supports it as well, but at the moment only if your OIDC provider can generate ID tokens in JWT format (which is very common this days). -See our [JWT Scope docs]({{< ref "/api-management/client-authentication#use-json-web-tokens-jwt" >}}) for more details. +See our [JWT Scope docs]({{< ref "api-management/client-authentication#use-json-web-tokens-jwt" >}}) for more details. ### Go plugins @@ -912,7 +912,7 @@ Here are the packages and their versions we are releasing today: Tyk Gateway v2. #### Mutual TLS -A major feature of this release is the implementation of Mutual TLS. Now you can protect your APIs by allow listing certificates, idenitfy users based on them, and increase security between Tyk and upstream API. For details, see [Mutual TLS]({{< ref "/api-management/client-authentication#use-mutual-tls" >}}). +A major feature of this release is the implementation of Mutual TLS. Now you can protect your APIs by allow listing certificates, idenitfy users based on them, and increase security between Tyk and upstream API. For details, see [Mutual TLS]({{< ref "api-management/client-authentication#use-mutual-tls" >}}). #### Extended use of Multiple Policies @@ -974,7 +974,7 @@ More SSO functionality is something that a lot of our customers have been asking ### Tyk Gateway v2.4.0 #### Mutual TLS support -[Docs]({{< ref "/api-management/client-authentication#use-mutual-tls" >}}) +[Docs]({{< ref "api-management/client-authentication#use-mutual-tls" >}}) #### Global API rate limits [Docs]({{< ref "api-management/rate-limit#rate-limiting-layers" >}}) @@ -1059,7 +1059,7 @@ This was very resource consuming and unstable feature. We recommend using load b ### Tyk Dashboard v1.4.0 #### Mutual TLS support -[Docs]({{< ref "/api-management/client-authentication#use-mutual-tls" >}}) +[Docs]({{< ref "api-management/client-authentication#use-mutual-tls" >}}) #### Global API rate limits [Docs]({{< ref "api-management/rate-limit#rate-limiting-layers" >}}) @@ -1163,7 +1163,7 @@ This is a UI only fix, it is still allowable via the API (which is OK). See https://tyk.io/docs/configure/tyk-pump-configuration/ for a sample pump.conf file. ### MDCB v1.4.0 -Added support for Mutual TLS, mentioned by Gateway and Dashboard above. See [Docs]({{< ref "/api-management/client-authentication#use-mutual-tls" >}}) +Added support for Mutual TLS, mentioned by Gateway and Dashboard above. See [Docs]({{< ref "api-management/client-authentication#use-mutual-tls" >}}) Also fixed bug when Mongo connections became growing though the roof if client with wrong credentials tries to connect. diff --git a/tyk-docs/content/developer-support/release-notes/dashboard.md b/tyk-docs/content/developer-support/release-notes/dashboard.md index 6612ff674e..55a19d973f 100644 --- a/tyk-docs/content/developer-support/release-notes/dashboard.md +++ b/tyk-docs/content/developer-support/release-notes/dashboard.md @@ -828,7 +828,7 @@ If you are upgrading to 5.5.0, please follow the detailed [upgrade instructions] docker pull tykio/tyk-dashboard:v5.5.0 ``` - Helm charts - - [tyk-charts v1.6]({{< ref "/developer-support/release-notes/helm-chart#160-release-notes" >}}) + - [tyk-charts v1.6]({{< ref "developer-support/release-notes/helm-chart#160-release-notes" >}}) #### Changelog {#Changelog-v5.5.0} -For MongoDB users: Tyk Charts 1.3.0 uses `mongo-go` as the default driver to connect to MongoDB. `mongo-go` driver is compatible with MongoDB 4.4.x and above. For MongoDB versions prior to 4.4, please set `global.mongo.driver` to `mgo`. We recommend reading [Choose a MongoDB driver]({{}}) when you need to change driver setting. +For MongoDB users: Tyk Charts 1.3.0 uses `mongo-go` as the default driver to connect to MongoDB. `mongo-go` driver is compatible with MongoDB 4.4.x and above. For MongoDB versions prior to 4.4, please set `global.mongo.driver` to `mgo`. We recommend reading [Choose a MongoDB driver]({{}}) when you need to change driver setting. @@ -1706,7 +1706,7 @@ For a comprehensive list of changes, please refer to the detailed [changelog]({{ Tyk Charts 1.3 adds support for a number of new Tyk features available from Tyk 5.3.0. These include: Support use of SSL certificates when connecting to Redis, Configurations for OAS Validate examples and OAS Validate Schema defaults. ##### Graph Pump -Tyk Charts 1.3 adds support for Graph MongoDB Pump, Graph SQL Pump and Graph SQL Aggregate Pump. see [Graph Pump setup]({{}}) to learn more about the GraphQL-specific metrics available. +Tyk Charts 1.3 adds support for Graph MongoDB Pump, Graph SQL Pump and Graph SQL Aggregate Pump. see [Graph Pump setup]({{}}) to learn more about the GraphQL-specific metrics available. ##### Enable Tyk Identity Broker (TIB) in Tyk Dashboard Tyk Charts 1.3 adds a field to enable Internal [Tyk Identity Broker (TIB)]({{}}) in Tyk Dashboard by field `tyk-dashboard.tib.enabled` to `true`. diff --git a/tyk-docs/content/developer-support/release-notes/portal.md b/tyk-docs/content/developer-support/release-notes/portal.md index edeb003b21..30843d2c6f 100644 --- a/tyk-docs/content/developer-support/release-notes/portal.md +++ b/tyk-docs/content/developer-support/release-notes/portal.md @@ -62,7 +62,7 @@ There are no deprecations in this release. If you are on 1.12.0 or an older version, we advise you to upgrade ASAP to this release. While upgrading to 1.13.0, Portal will automatically migrate the new Custom IDs to most of the existing resources. For more information, please refer to the [changelog](#Changelog-v1.13.0). -To upgrade the portal's theme, please follow the [upgrade instructions]({{< ref "product-stack/tyk-enterprise-developer-portal/upgrading/theme-upgrades" >}}) for the portal's themes. +To upgrade the portal's theme, please follow the [upgrade instructions]({{< ref "portal/customization#upgrading-themes" >}}) for the portal's themes. #### Download - [Docker image v1.13.0](https://hub.docker.com/r/tykio/portal/tags?page=&page_size=&ordering=&name=v1.13.0) @@ -273,7 +273,7 @@ There are no deprecations in this release. #### Upgrade instructions If you are on 1.11.0 or an older version we advise you to upgrade ASAP directly to this release. -To upgrade the portal's theme please follow the [upgrade instructions]({{< ref "product-stack/tyk-enterprise-developer-portal/upgrading/theme-upgrades" >}}) for the portal's themes. +To upgrade the portal's theme please follow the [upgrade instructions]({{< ref "portal/customization#upgrading-themes" >}}) for the portal's themes. #### Download - [Docker image v1.12.0](https://hub.docker.com/r/tykio/portal/tags?page=&page_size=&ordering=&name=v1.12.0) @@ -314,7 +314,7 @@ We support out of the box integration with the following SSO providers type: - LDAP: Bind users to an LDAP server such as Azure Active Directory, using their username and password. - Social: The social provider should provide seamless integration with Google+ Github, Facebook, Salesforce, Digital Ocean and more. -You can read more about the supported SSO providers [here]({{< ref "/tyk-identity-broker" >}}). +You can read more about the supported SSO providers [here]({{< ref "tyk-identity-broker" >}}).
@@ -492,15 +492,15 @@ The v1.11.0 release includes the following new features and improvements: To improve stability under high loads, we conducted performance testing and identified that improper database configurations can cause unexpected portal restarts. To prevent this and ensure optimal performance, we recommend the following database settings: **Recommended Configuration:** -- [PORTAL_DATABASE_MAX_OPEN_CONNECTIONS]({{< ref "/product-stack/tyk-enterprise-developer-portal/deploy/configuration#portal_database_max_open_connections" >}}): Set this value based on your database's maximum connection limit divided by the number of portal instances. For example, if your database allows 200 connections and you are running 4 portal instances, set PORTAL_DATABASE_MAX_OPEN_CONNECTIONS to 50 per instance. This ensures that all instances can share the available connections without exceeding the database's limit, which could otherwise lead to performance degradation or errors. -- [PORTAL_DATABASE_MAX_IDLE_CONNECTIONS]({{< ref "/product-stack/tyk-enterprise-developer-portal/deploy/configuration#portal_database_max_idle_connections" >}}): Set to 15 or a lower value based on your expected load. This setting keeps a reasonable number of connections readily available without tying up resources unnecessarily. +- [PORTAL_DATABASE_MAX_OPEN_CONNECTIONS]({{< ref "product-stack/tyk-enterprise-developer-portal/deploy/configuration#portal_database_max_open_connections" >}}): Set this value based on your database's maximum connection limit divided by the number of portal instances. For example, if your database allows 200 connections and you are running 4 portal instances, set PORTAL_DATABASE_MAX_OPEN_CONNECTIONS to 50 per instance. This ensures that all instances can share the available connections without exceeding the database's limit, which could otherwise lead to performance degradation or errors. +- [PORTAL_DATABASE_MAX_IDLE_CONNECTIONS]({{< ref "product-stack/tyk-enterprise-developer-portal/deploy/configuration#portal_database_max_idle_connections" >}}): Set to 15 or a lower value based on your expected load. This setting keeps a reasonable number of connections readily available without tying up resources unnecessarily. For reference, with 2 portal instances, `PORTAL_DATABASE_MAX_OPEN_CONNECTIONS` set to 30 and `PORTAL_DATABASE_MAX_IDLE_CONNECTIONS` set to 15, we could handle 90 active users. #### Upgrade instructions If you are on 1.10.0 or an older version we advise you to upgrade ASAP directly to this release. -To upgrade the portal's theme please follow the [upgrade instructions]({{< ref "product-stack/tyk-enterprise-developer-portal/upgrading/theme-upgrades" >}}) for the portal's themes. +To upgrade the portal's theme please follow the [upgrade instructions]({{< ref "portal/customization#upgrading-themes" >}}) for the portal's themes. #### Download - [Docker image v1.11.0](https://hub.docker.com/r/tykio/portal/tags?page=&page_size=&ordering=&name=v1.11.0) @@ -519,14 +519,14 @@ To upgrade the portal's theme please follow the [upgrade instructions]({{< ref " - Added APIs to manage blog posts along with their tags and categories. - Added a new API endpoint that allows the rotation of API credentials. - UI and API for themes soft delete. Soft deleted themes are not shown in the UI and API, but are kept in the database for future reference. -- Added new TLS variables to set MinVersion ([portal_tls_min_version]({{< ref "/product-stack/tyk-enterprise-developer-portal/deploy/configuration#portal_tls_min_version" >}}), MaxVersion ([PORTAL_TLSCONFIG_MAXVERSION]({{< ref "/product-stack/tyk-enterprise-developer-portal/deploy/configuration#portal_tls_max_version" >}}), and CipherSuites ([PORTAL_TLS_CIPHER_SUITES]({{< ref "/product-stack/tyk-enterprise-developer-portal/deploy/configuration#portal_tls_cipher_suites" >}}). -- Added a new configuration to manage the idle timeout of the portal's session ([PORTAL_SESSION_IDLE_TIMEOUT]({{< ref "/product-stack/tyk-enterprise-developer-portal/deploy/configuration#portal_session_idle_timeout" >}}). +- Added new TLS variables to set MinVersion ([portal_tls_min_version]({{< ref "product-stack/tyk-enterprise-developer-portal/deploy/configuration#portal_tls_min_version" >}}), MaxVersion ([PORTAL_TLSCONFIG_MAXVERSION]({{< ref "product-stack/tyk-enterprise-developer-portal/deploy/configuration#portal_tls_max_version" >}}), and CipherSuites ([PORTAL_TLS_CIPHER_SUITES]({{< ref "product-stack/tyk-enterprise-developer-portal/deploy/configuration#portal_tls_cipher_suites" >}}). +- Added a new configuration to manage the idle timeout of the portal's session ([PORTAL_SESSION_IDLE_TIMEOUT]({{< ref "product-stack/tyk-enterprise-developer-portal/deploy/configuration#portal_session_idle_timeout" >}}). - Added CSRF protection injection to portal's form. Now you don't need to add it manually to your templates. ##### Changed - Changed passwordrecovery links to be valid for 24 hours. - Changed password recovery links to be unique and valid for one use only. -- Changed the default value of [PORTAL_DATABASE_CONNECTION_MAX_LIFETIME]({{< ref "/product-stack/tyk-enterprise-developer-portal/deploy/configuration#portal_database_connection_max_lifetime" >}}) to 1800000 milliseconds. +- Changed the default value of [PORTAL_DATABASE_CONNECTION_MAX_LIFETIME]({{< ref "product-stack/tyk-enterprise-developer-portal/deploy/configuration#portal_database_connection_max_lifetime" >}}) to 1800000 milliseconds. - Changed session token queries for better performance. - Introduced new indexes for better performance. @@ -597,7 +597,7 @@ The 1.10.0 addresses twenty high-priority bugs and vulnerabilities and introduce #### Upgrade instructions If you are on 1.9.0 or an older version we advise you to upgrade ASAP directly to this release. -To upgrade the portal's theme please follow the [upgrade instructions]({{< ref "product-stack/tyk-enterprise-developer-portal/upgrading/theme-upgrades" >}}) for the portal's themes. +To upgrade the portal's theme please follow the [upgrade instructions]({{< ref "portal/customization#upgrading-themes" >}}) for the portal's themes. #### Download - [Docker image v1.10.0](https://hub.docker.com/r/tykio/portal/tags?page=&page_size=&ordering=&name=v1.10.0) @@ -609,8 +609,8 @@ To upgrade the portal's theme please follow the [upgrade instructions]({{< ref " #### Changelog ##### Added - Added OAS APIs support. -- Added an assets cache for improved performance on database-backed themes. This speeds up the portal's pages loading time by 30%. It's enabled by default and you can disable using [PORTAL_ASSETS_CACHE_DISABLE]({{< ref "/product-stack/tyk-enterprise-developer-portal/deploy/configuration#portal_assets_cache_disable" >}}). -- Added three new configuration options to manage database connections lifecycle: [PORTAL_DATABASE_MAX_OPEN_CONNECTIONS]({{< ref "/product-stack/tyk-enterprise-developer-portal/deploy/configuration#portal_database_max_open_connections" >}}), [PORTAL_DATABASE_MAX_IDLE_CONNECTIONS]({{< ref "/product-stack/tyk-enterprise-developer-portal/deploy/configuration#portal_database_max_idle_connections" >}}), and [PORTAL_DATABASE_CONNECTION_MAX_LIFETIME]({{< ref "/product-stack/tyk-enterprise-developer-portal/deploy/configuration#portal_database_connection_max_lifetime" >}}). +- Added an assets cache for improved performance on database-backed themes. This speeds up the portal's pages loading time by 30%. It's enabled by default and you can disable using [PORTAL_ASSETS_CACHE_DISABLE]({{< ref "product-stack/tyk-enterprise-developer-portal/deploy/configuration#portal_assets_cache_disable" >}}). +- Added three new configuration options to manage database connections lifecycle: [PORTAL_DATABASE_MAX_OPEN_CONNECTIONS]({{< ref "product-stack/tyk-enterprise-developer-portal/deploy/configuration#portal_database_max_open_connections" >}}), [PORTAL_DATABASE_MAX_IDLE_CONNECTIONS]({{< ref "product-stack/tyk-enterprise-developer-portal/deploy/configuration#portal_database_max_idle_connections" >}}), and [PORTAL_DATABASE_CONNECTION_MAX_LIFETIME]({{< ref "product-stack/tyk-enterprise-developer-portal/deploy/configuration#portal_database_connection_max_lifetime" >}}). ##### Fixed - Fixed the bug where `PORTAL_SESSION_LIFETIME` was calculated in minutes instead of seconds. @@ -655,7 +655,7 @@ There are no deprecations in this release. #### Release Highlights The 1.9.0 release addresses several security vulnerability and bugs and introduces two new capabilities: -- [Webhooks]({{< ref "product-stack/tyk-enterprise-developer-portal/portal-customisation/configure-webhooks.md" >}}) for events that happen in the portal. +- [Webhooks]({{< ref "portal/customization" >}}) for events that happen in the portal. - [Admin APIs]({{< ref "product-stack/tyk-enterprise-developer-portal/api-documentation/tyk-edp-api" >}}) for OAuth2.0 configuration. #### Upgrade instructions @@ -672,7 +672,7 @@ This release doesn't introduce any changes to the theme, so a theme upgrade is n #### Changelog ##### Added -- Added [the webhooks]({{< ref "product-stack/tyk-enterprise-developer-portal/portal-customisation/configure-webhooks.md" >}}) capability that enable real-time, automated data updates between the portal and 3rd party applications. +- Added [the webhooks]({{< ref "portal/customization" >}}) capability that enable real-time, automated data updates between the portal and 3rd party applications. - Added [admin APIs]({{< ref "product-stack/tyk-enterprise-developer-portal/api-documentation/tyk-edp-api" >}}) for managing OAuth2.0 configuration. ##### Fixed @@ -710,7 +710,7 @@ If you are not on v1.8.4 then there's no urgency in updating. #### Upgrade instructions If you are on 1.8.4 you should **upgrade ASAP** directly to this release. This release doesn't introduce any changes to the theme, so a theme upgrade is not required. -If you are on 1.8.3 or older version please follow the [upgrade instructions]({{< ref "product-stack/tyk-enterprise-developer-portal/upgrading/theme-upgrades" >}}) to upgrade the portal's themes. +If you are on 1.8.3 or older version please follow the [upgrade instructions]({{< ref "portal/customization#upgrading-themes" >}}) to upgrade the portal's themes. #### Download - [Docker image v1.8.5](https://hub.docker.com/r/tykio/portal/tags?page=&page_size=&ordering=&name=v1.8.5) @@ -740,7 +740,7 @@ There are no deprecations in this release. #### Upgrade instructions If you are on 1.8.1 or an older version we advise you to upgrade ASAP directly to this release. -To upgrade the portal's theme please follow the [upgrade instructions]({{< ref "product-stack/tyk-enterprise-developer-portal/upgrading/theme-upgrades" >}}) for the portal's themes. +To upgrade the portal's theme please follow the [upgrade instructions]({{< ref "portal/customization#upgrading-themes" >}}) for the portal's themes. #### Release Highlights @@ -776,15 +776,15 @@ The 1.8.4 release addresses ten high-priority bugs and vulnerabilities, and intr | /portal/private/profile | Profile | | /auth/password/login | Developer portal login | | /auth/password/new | Password reset | -- Changed the credential provisioning flow to automatically include DeveloperID, OrganizationID, ApplicationID, and TeamIDs in [the credential metadata]({{< ref "/product-stack/tyk-enterprise-developer-portal/portal-customisation/customise-user-model#default-attributes" >}}). -- Added warning regarding potential PII exposure to the [custom attributes menu]({{< ref "/product-stack/tyk-enterprise-developer-portal/portal-customisation/customise-user-model#default-attributes" >}}). +- Changed the credential provisioning flow to automatically include DeveloperID, OrganizationID, ApplicationID, and TeamIDs in [the credential metadata]({{< ref "portal/customization#default-attributes-of-user-model" >}}). +- Added warning regarding potential PII exposure to the [custom attributes menu]({{< ref "portal/customization#default-attributes-of-user-model" >}}). - Changed the behavior of the portal for 404 errors. Now whenever a user requests non-existing page both private (e.i. requiring sign-in to access) or public, the portal now always renders the `not_found.tmpl` template. -- Changed the behavior of the `Secure` cookie attribute that is set by [PORTAL_SESSION_SECURE]({{< ref "/product-stack/tyk-enterprise-developer-portal/deploy/configuration#portal_session_secure" >}}) so that the `Secure` attribute is always add to the `Set-Cookie` header whenever `PORTAL_SESSION_SECURE` is set to `true` or when TLS is enabled. +- Changed the behavior of the `Secure` cookie attribute that is set by [PORTAL_SESSION_SECURE]({{< ref "product-stack/tyk-enterprise-developer-portal/deploy/configuration#portal_session_secure" >}}) so that the `Secure` attribute is always add to the `Set-Cookie` header whenever `PORTAL_SESSION_SECURE` is set to `true` or when TLS is enabled. - Changed the behavior of removing a developer profile within the developers UI in the admin app. Now, when an admin tries to remove a developer profile and some of their credentials have been removed from the credentials provider, or if the provider itself is down or unreachable, the portal asks the admin if they still want to remove the developer profile by displaying a modal window. -- Extended the `DELETE /users/:id` API endpoint by adding the [?force]({{< ref "/product-stack/tyk-enterprise-developer-portal/api-documentation/tyk-edp-api" >}}) query parameter to force removal of a user even if some of their credentials have been removed from the credentials provider, or if the provider itself is down or unreachable. +- Extended the `DELETE /users/:id` API endpoint by adding the [?force]({{< ref "product-stack/tyk-enterprise-developer-portal/api-documentation/tyk-edp-api" >}}) query parameter to force removal of a user even if some of their credentials have been removed from the credentials provider, or if the provider itself is down or unreachable. - Extended the `GET /pages/:id/content-blocks/:id:` API endpoint by adding additional fields in the response body: `Content`, `MarkdownContent`, `MarkdownEnabled`, `Name`, and `PageID`. -- Extended [filesize limit]({{< ref "tyk-stack/tyk-developer-portal/enterprise-developer-portal/customise-enterprise-portal/full-customisation/file-structure-concepts#part-1-create-a-new-theme" >}}) for individual files in themes to 5 MB. -- Made the organization invite email's subject configurable via [the emails settings section]({{< ref "tyk-stack/tyk-developer-portal/enterprise-developer-portal/customise-enterprise-portal/full-customisation/email-customization#list-of-email-notifications" >}}). +- Extended [filesize limit]({{< ref "portal/customization#create-a-theme" >}}) for individual files in themes to 5 MB. +- Made the organization invite email's subject configurable via [the emails settings section]({{< ref "portal/customization#supported-email-notifications" >}}). ##### Fixed - Fixed the bug where it was impossible to create an ordered list in the rich text editor in the admin app due to CSS issues. @@ -812,7 +812,7 @@ There are no deprecations in this release. #### Upgrade instructions If you are on 1.8.1 or an older version we advise you to upgrade ASAP directly to this release. -To upgrade the portal's theme please follow the [upgrade instructions]({{< ref "product-stack/tyk-enterprise-developer-portal/upgrading/theme-upgrades" >}}) for the portal's themes. +To upgrade the portal's theme please follow the [upgrade instructions]({{< ref "portal/customization#upgrading-themes" >}}) for the portal's themes. #### Release Highlights @@ -855,7 +855,7 @@ There are no deprecations in this release. #### Upgrade instructions If you are on 1.8.1 or an older version we advise you to upgrade ASAP directly to this release. -To upgrade the portal's theme please follow the [upgrade instructions]({{< ref "product-stack/tyk-enterprise-developer-portal/upgrading/theme-upgrades" >}}) for the portal's themes. +To upgrade the portal's theme please follow the [upgrade instructions]({{< ref "portal/customization#upgrading-themes" >}}) for the portal's themes. #### Release Highlights @@ -902,7 +902,7 @@ There are no deprecations in this release. If you are on 1.8.0 or an older version we advise you to upgrade ASAP directly to this release. Unlike 1.8.0, 1.8.1 fixes the broken backward compatability for the default visual theme. Therefore, the upgrade path from earlier versions are straightforward. It is enough to just pull the latest version of the portal's container. -To upgrade the portal's theme please follow the [upgrade instructions]({{< ref "product-stack/tyk-enterprise-developer-portal/upgrading/theme-upgrades" >}}) for the portal's themes. +To upgrade the portal's theme please follow the [upgrade instructions]({{< ref "portal/customization#upgrading-themes" >}}) for the portal's themes. #### Release Highlights @@ -953,7 +953,7 @@ There are no deprecations in this release. #### Upgrade instructions If you are on 1.7.0 or an older version we advise you to upgrade ASAP directly to this release. When upgrading from 1.6.0 or earlier versions, customers may experience problems when starting the portal. One of the possible issues is the following: -- When the portal theme [manifest]({{< ref "tyk-stack/tyk-developer-portal/enterprise-developer-portal/customise-enterprise-portal/full-customisation/file-structure-concepts#manifest-file" >}}) has a reference to a template that is not present in the theme then the theme won't be loaded. This check that prevents admin users from uploading themes with potential errors was introduced in version [1.7.0]({{< ref "#content-blocks-validation" >}}). +- When the portal theme [manifest]({{< ref "portal/customization#manifest-file" >}}) has a reference to a template that is not present in the theme then the theme won't be loaded. This check that prevents admin users from uploading themes with potential errors was introduced in version [1.7.0]({{< ref "#content-blocks-validation" >}}). - At the same time, the default theme in version 1.6.0 of the portal had a reference in the theme manifest to the `portal_home` template that didn't exist in the theme. - The portal doesn't update the theme automatically because in that case any customer-made changes will be lost. Subsequently, upgrading from 1.6.0 to 1.8.0 may result in the following error when loading the theme: ```yaml @@ -1038,7 +1038,7 @@ Extend the User model with additional fields of one of four types: - Determine if developers can view the fields or are they restricted to only admin users? - Can developers edit the fields? -All settings are available via the [admin API]({{< ref "product-stack/tyk-enterprise-developer-portal/api-documentation/tyk-edp-api.md" >}}) and the UI. +All settings are available via the [admin API]({{< ref "product-stack/tyk-enterprise-developer-portal/api-documentation/tyk-edp-api" >}}) and the UI. To create a custom attribute, define it in the custom attributes menu: {{< img src="/img/dashboard/portal-management/enterprise-portal/1.8.0-create-custom-attribute.png" width=500px alt="Create a custom attribute for the User model">}} @@ -1088,7 +1088,7 @@ Now when the [PORTAL_DCR_LOG_ENABLED]({{< ref "product-stack/tyk-enterprise-deve ##### Fixed - Fixed the bug where the database credentials were printed in the logs when bootstrapping the portal. - Fixed the bug where the session cookie was disclosing the username and role. -- Fixed the bug where the [Forgot Password page]({{< ref "tyk-developer-portal/tyk-enterprise-developer-portal/api-consumer-portal/reset-password.md#introduction" >}}) did not reflect the current theme. +- Fixed the bug where the [Forgot Password page]({{< ref "portal/api-consumer#reset-password" >}}) did not reflect the current theme. - Fixed the bug where the DCR flow failed to create a client with policies managed by Tyk Operator. - Fixed the bug where an admin user couldn't upload a new theme file in Kubernetes environment. - Fixed the bug where the portal application went down after running for several hours in Kubernetes environment. @@ -1325,7 +1325,7 @@ Until recently, SQL storage was not supported for the portal's assets: OAS files - Filesystem or S3 for assets (pictures, themes, etc). This is especially inconvenient in Kubernetes environment when customers had to use persistent volumes. -With this new feature, customers can simply use the same SQL database (MySQL, MariaDB and PostgreSQL) for both assets and metadata. To use the `db` [storage type]({{< ref "/content/product-stack/tyk-enterprise-developer-portal/deploy/configuration.md#portal_storage" >}}) just set the `PORTAL_STORAGE=db` for environment variables or `"Storage": "db"` in a config file and you are good to go! +With this new feature, customers can simply use the same SQL database (MySQL, MariaDB and PostgreSQL) for both assets and metadata. To use the `db` [storage type]({{< ref "product-stack/tyk-enterprise-developer-portal/deploy/configuration.md#portal_storage" >}}) just set the `PORTAL_STORAGE=db` for environment variables or `"Storage": "db"` in a config file and you are good to go! ##### Response status code added to API analytics filters API Consumers now can filter API analytics by response status codes. This allows them to analyze traffic and error rate for specific response code for their API Products. @@ -1430,10 +1430,10 @@ If you are on a 1.1.0 or an older version we advise you to upgrade ASAP directly This release is primarily focused on improved deployment support for Kubernetes and a variety of features to achieve better developer experience. ##### Full Kubernetes support -The Tyk Enterprise Developer Portal is now available in Kubernetes and customer can launch it using our [helm charts]({{}}). This feature makes the portal Kubernetes friendly by adding liveness, readiness probes, graceful shutdown and changing the portal lifecycle so that it's possible to set an initial user and bootstrap the portal via APIs. +The Tyk Enterprise Developer Portal is now available in Kubernetes and customer can launch it using our [helm charts]({{}}). This feature makes the portal Kubernetes friendly by adding liveness, readiness probes, graceful shutdown and changing the portal lifecycle so that it's possible to set an initial user and bootstrap the portal via APIs. ##### SSO for API Consumers and admins -API Providers can [configure Single Sign-on]({{}}) for the Enterprise developer portal so that it's possible to login developers and admins to the portal user 3rd party IdP. +API Providers can [configure Single Sign-on]({{}}) for the Enterprise developer portal so that it's possible to login developers and admins to the portal user 3rd party IdP. ##### API Analytics for API Consumers This capability enables API Providers to get aggregated statistics about consumption of their APIs using Tyk Pump. In 1.2.0, we enabled the portal to attach the following tags to API Keys and oAuth clients: @@ -1458,12 +1458,12 @@ This new setting allows API Providers to set the logging [level]({{< ref "produc #### Changelog ##### Added -- Added Kubernetes support and [helm charts]({{< ref "product-stack/tyk-enterprise-developer-portal/deploy/install-tyk-enterprise-portal/install-portal-using-helm" >}}). -- Added [Single Sign-on]({{}}) for API Consumers and admin users so that they can use their IdPs for managing admin users and developers. +- Added Kubernetes support and [helm charts]({{< ref "portal/install#using-legacy-helm-chart" >}}). +- Added [Single Sign-on]({{}}) for API Consumers and admin users so that they can use their IdPs for managing admin users and developers. - Added organization and application metadata to auth tokens and OAuth2.0 clients so that API Providers can use Tyk Pump to create aggregated reports based on the metadata from tokens and OAuth2.0 clients. - Added Admin APIs for API Products to enable API Providers to update API Products using CI/CD pipelines. - Added [TLS]({{}}) support for the portal's UI. -- Added config options to set the logging [level]({{< ref "product-stack/tyk-enterprise-developer-portal/deploy/configuration#portal_log_level" >}}) and [format]({{< ref "/product-stack/tyk-enterprise-developer-portal/deploy/configuration#portal_log_format" >}}). This offers API Providers more control over the logging behavior of their APIs. +- Added config options to set the logging [level]({{< ref "product-stack/tyk-enterprise-developer-portal/deploy/configuration#portal_log_level" >}}) and [format]({{< ref "product-stack/tyk-enterprise-developer-portal/deploy/configuration#portal_log_format" >}}). This offers API Providers more control over the logging behavior of their APIs. ##### Fixed @@ -1491,7 +1491,7 @@ We advise you to upgrade ASAP directly to this release. This release introduce a variety of features to improve developer experience. Additionally, we've included support for the S3 storage type as well as some bug fixes. ##### Organization management for API Consumers -Now API Consumers can [create organizations]({{}}) and securely share credentials between their teammates. In greater detail: +Now API Consumers can [create organizations]({{}}) and securely share credentials between their teammates. In greater detail: - API Consumers can request to upgrade their account to an organizational account. - API Consumers can invite teammates to their organization and manage their roles. - API Consumers in the same organization share access credentials so that the API Consumer team will still have access to API credentials even if an admin user is on vacation or left the organization. @@ -1499,12 +1499,12 @@ Now API Consumers can [create organizations]({{}}) to API Products for better developer experiences: +API Providers can add [Get started guides]({{}}) to API Products for better developer experiences: - API Providers can add the **Get started guides** to API Products to speed-up onboarding of API Consumers. - API Providers can use HTML or Markdown editors for authoring content for API Consumers such as the Get started guides and blog posts. ##### Tags for API Products and blog posts -API Providers can select which blogs posts to display on an API Product page using [the tags feature]({{}}). To achieve that, an API Provider can specify tags for both API Products and blog posts. Blog posts that match tags with an API Product are displayed in the 'Related blog content' section in the API Product page. This offers API Providers greater control over what blog posts to display on their API Product page. +API Providers can select which blogs posts to display on an API Product page using [the tags feature]({{}}). To achieve that, an API Provider can specify tags for both API Products and blog posts. Blog posts that match tags with an API Product are displayed in the 'Related blog content' section in the API Product page. This offers API Providers greater control over what blog posts to display on their API Product page. ##### S3 support We added [S3 support]({{}}) for the portal assets storage (themes, images, OAS files). This update enhances the extensibility of our platform, allowing you to choose different storage solutions to better align with your specific needs. @@ -1515,10 +1515,10 @@ We added [S3 support]({{}}) for API Consumers to safely share API access credentials between team members. -- Added the [Get started guides]({{}}) for API Products so that admins can explain to their consumers how use their API Products. +- Added the [organization management capability]({{}}) for API Consumers to safely share API access credentials between team members. +- Added the [Get started guides]({{}}) for API Products so that admins can explain to their consumers how use their API Products. - Added support for [S3 storage]({{}}) for the portal's assets storage. Now our customers can use `s3` storage in addition to the filesystem which is especially important in Kubernetes environments. -- Added [tags]({{}}) for API Products and blog posts so that API Providers have greater control over which blog posts to display on their API Product page. +- Added [tags]({{}}) for API Products and blog posts so that API Providers have greater control over which blog posts to display on their API Product page. ##### Fixed diff --git a/tyk-docs/content/developer-support/release-notes/tib.md b/tyk-docs/content/developer-support/release-notes/tib.md index 69c786d0c2..130d7fc119 100644 --- a/tyk-docs/content/developer-support/release-notes/tib.md +++ b/tyk-docs/content/developer-support/release-notes/tib.md @@ -115,7 +115,7 @@ Each change log item should be expandable. The first line summarises the changel This release adds support for JSON Web Encryption (JWE) in OIDC Single Sign-On (SSO) with TIB, providing enhanced security for token handling in authentication flows. This feature enables processing and validation of JWE tokens, with configuration options for setting the private key required for decryption. -For more details, refer to the [OIDC SSO with JWE]({{}}) documentation. +For more details, refer to the [OIDC SSO with JWE]({{}}) documentation. diff --git a/tyk-docs/content/getting-started.md b/tyk-docs/content/getting-started.md index a75df732a1..3703d7e4e3 100755 --- a/tyk-docs/content/getting-started.md +++ b/tyk-docs/content/getting-started.md @@ -22,7 +22,7 @@ Not sure what you need? Visit our [APIM comparison]({{< ref "apim" >}}) page to The following tutorials will show you how to get up and running with Tyk depending on your installation type. -### [Create an API]({{< ref "/getting-started/create-api" >}}) +### [Create an API]({{< ref "getting-started/create-api" >}}) How to add a basic API to Tyk diff --git a/tyk-docs/content/getting-started/installation.md b/tyk-docs/content/getting-started/installation.md index e636e5d4dd..08b4986608 100644 --- a/tyk-docs/content/getting-started/installation.md +++ b/tyk-docs/content/getting-started/installation.md @@ -71,7 +71,7 @@ Install and manage Tyk yourself with our open-source API Gateway. ## Not Sure Which to Choose? -Visit our [Comparison Page]({{< ref "/apim" >}}) to understand the different installation types and their benefits. +Visit our [Comparison Page]({{< ref "apim" >}}) to understand the different installation types and their benefits. Also, feel free to: diff --git a/tyk-docs/content/getting-started/key-concepts/graphql-subscriptions.md b/tyk-docs/content/getting-started/key-concepts/graphql-subscriptions.md index ad8f610809..19a1435dcd 100644 --- a/tyk-docs/content/getting-started/key-concepts/graphql-subscriptions.md +++ b/tyk-docs/content/getting-started/key-concepts/graphql-subscriptions.md @@ -97,7 +97,7 @@ If the upstream subscription GraphQL API is protected please enable the authenti {{< /note >}} -There is no need to enable subscriptions separately. They are supported alongside GraphQL as a standard. The only requirement for subscriptions to work is to [enable WebSockets]({{< ref "graphql/graphql-websockets.md" >}}) in your Tyk Gateway configuration file. +There is no need to enable subscriptions separately. They are supported alongside GraphQL as a standard. The only requirement for subscriptions to work is to [enable WebSockets]({{< ref "graphql/graphql-websockets" >}}) in your Tyk Gateway configuration file. Here's a general sequence diagram showing how subscriptions in Tyk work exactly: diff --git a/tyk-docs/content/getting-started/key-concepts/high-level-concepts.md b/tyk-docs/content/getting-started/key-concepts/high-level-concepts.md index 8997571b44..384c52732f 100644 --- a/tyk-docs/content/getting-started/key-concepts/high-level-concepts.md +++ b/tyk-docs/content/getting-started/key-concepts/high-level-concepts.md @@ -85,7 +85,7 @@ We provide a flexible way for you to [export a Tyk OAS API definition]({{< ref " Your OpenAPI description is a living document that describes your upstream service. When this is updated (for example, due to the addition of a new endpoint) instead of having to create a new Tyk OAS API to expose this, you can easily [update the OpenAPI]({{< ref "getting-started/using-oas-definitions/update-an-oas-api" >}}) part of your Tyk OAS API with the new OpenAPI document. -When you need to make breaking changes as your services and APIs evolve, it's easy to [use versioning with Tyk OAS APIs]({{< ref "/getting-started/using-oas-definitions/versioning-an-oas-api" >}}). +When you need to make breaking changes as your services and APIs evolve, it's easy to [use versioning with Tyk OAS APIs]({{< ref "getting-started/using-oas-definitions/versioning-an-oas-api" >}}). ## Community Feedback diff --git a/tyk-docs/content/getting-started/using-oas-definitions/import-an-oas-api.md b/tyk-docs/content/getting-started/using-oas-definitions/import-an-oas-api.md index c4bc7cacc5..58b83972c4 100644 --- a/tyk-docs/content/getting-started/using-oas-definitions/import-an-oas-api.md +++ b/tyk-docs/content/getting-started/using-oas-definitions/import-an-oas-api.md @@ -24,7 +24,7 @@ You can also run these steps using the Tyk Dashboard API, noting the differences | Tyk Gateway API | 8080 | `tyk/apis/oas` | `x-tyk-authorization` | `secret` value set in `tyk.conf` | | Tyk Dashboard API | 3000 | `api/apis/oas` | `Authorization` | From Dashboard User Profile | -As explained in the section on [Creating an OAS API]({{< ref "/getting-started/using-oas-definitions/create-an-oas-api" >}}) remember that when using the Tyk Dashboard API you only need to issue one command to create the API and load it onto the Gateway; when using the Tyk Gateway API you must remember to restart or hot reload the Gateway after creating the API. +As explained in the section on [Creating an OAS API]({{< ref "getting-started/using-oas-definitions/create-an-oas-api" >}}) remember that when using the Tyk Dashboard API you only need to issue one command to create the API and load it onto the Gateway; when using the Tyk Gateway API you must remember to restart or hot reload the Gateway after creating the API. * When using the Tyk Dashboard API, you can find your credentials key from your **User Profile > Edit Profile > Tyk Dashboard API Access Credentials** @@ -895,7 +895,7 @@ Tyk supports the following options when importing an API: - From an existing Tyk API definition (Classic or OAS) - From a SOAP WSDL definition -The process for importing from an existing Tyk API definition or SOAP WSDL definition is explained [here]({{< ref "/getting-started/import-apis#import-apis-via-the-dashboard-api" >}}). The import function will now accept an existing Tyk OAS API definition, the process is the same as for a Tyk Classic API definition. +The process for importing from an existing Tyk API definition or SOAP WSDL definition is explained [here]({{< ref "getting-started/import-apis#import-apis-via-the-dashboard-api" >}}). The import function will now accept an existing Tyk OAS API definition, the process is the same as for a Tyk Classic API definition. #### Importing an OpenAPI Document diff --git a/tyk-docs/content/getting-started/using-oas-definitions/versioning-an-oas-api.md b/tyk-docs/content/getting-started/using-oas-definitions/versioning-an-oas-api.md index 23d7091727..3ee6ea04a2 100644 --- a/tyk-docs/content/getting-started/using-oas-definitions/versioning-an-oas-api.md +++ b/tyk-docs/content/getting-started/using-oas-definitions/versioning-an-oas-api.md @@ -35,7 +35,7 @@ You can also run these steps using the Tyk Dashboard API, noting the differences | Tyk Gateway API | 8080 | `tyk/apis/oas` | `x-tyk-authorization` | `secret` value set in `tyk.conf` | | Tyk Dashboard API | 3000 | `api/apis/oas` | `Authorization` | From Dashboard User Profile | -As explained in the section on [Creating an OAS API]({{< ref "/getting-started/using-oas-definitions/create-an-oas-api" >}}) remember that when using the Tyk Dashboard API you only need to issue one command to create the API and load it onto the Gateway; when using the Tyk Gateway API you must remember to restart or hot reload the Gateway after creating the API. +As explained in the section on [Creating an OAS API]({{< ref "getting-started/using-oas-definitions/create-an-oas-api" >}}) remember that when using the Tyk Dashboard API you only need to issue one command to create the API and load it onto the Gateway; when using the Tyk Gateway API you must remember to restart or hot reload the Gateway after creating the API. * When using the Tyk Dashboard API, you can find your credentials key from your **User Profile > Edit Profile > Tyk Dashboard API Access Credentials** diff --git a/tyk-docs/content/graphql.md b/tyk-docs/content/graphql.md index b20b87e3e7..4194d86424 100644 --- a/tyk-docs/content/graphql.md +++ b/tyk-docs/content/graphql.md @@ -18,9 +18,9 @@ This means support for the following operations: ## What can you do with GraphQL and Tyk? -You can securely expose existing GraphQL APIs using our [GraphQL core functionality]({{< ref "/content/graphql/creating-gql-api.md" >}}). +You can securely expose existing GraphQL APIs using our [GraphQL core functionality]({{< ref "graphql/creating-gql-api" >}}). -In addition to this, you can also use Tyk's integrated GraphQL engine to build a [Universal Data Graph]({{< ref "/content/universal-data-graph.md" >}}). The Universal Data Graph (UDG) lets you expose existing services as one single combined GraphQL API. +In addition to this, you can also use Tyk's integrated GraphQL engine to build a [Universal Data Graph]({{< ref "universal-data-graph" >}}). The Universal Data Graph (UDG) lets you expose existing services as one single combined GraphQL API. See our video on getting started with GraphQL. diff --git a/tyk-docs/content/graphql/creating-gql-api.md b/tyk-docs/content/graphql/creating-gql-api.md index 45ddb347ff..5d8d5373ec 100644 --- a/tyk-docs/content/graphql/creating-gql-api.md +++ b/tyk-docs/content/graphql/creating-gql-api.md @@ -11,7 +11,7 @@ GraphQL API can be created in Tyk using: * Tyk Dashboard API * Tyk Gateway API - for OSS users -The process is very similar to [HTTP API creation]({{< ref "/getting-started/create-api" >}}) with a few additional steps to cover GraphQL specific functionalities. +The process is very similar to [HTTP API creation]({{< ref "getting-started/create-api" >}}) with a few additional steps to cover GraphQL specific functionalities. {{< tabs_start >}} {{< tab_start "Via Tyk Dahsboard UI" >}} @@ -48,12 +48,12 @@ From the **Authentication** section: You have the following options: -- **Authentication mode**: This is the security method to use with your API. First, you can set it to `Open(Keyless)`, but that option is not advised for production APIs. See [Client Authentication]({{< ref "/api-management/client-authentication" >}}) for more details on securing your API. +- **Authentication mode**: This is the security method to use with your API. First, you can set it to `Open(Keyless)`, but that option is not advised for production APIs. See [Client Authentication]({{< ref "api-management/client-authentication" >}}) for more details on securing your API. - **Strip Authorization Data**: Select this option to strip any authorization data from your API requests. - **Auth Key Header Name**: The header name that will hold the token on inbound requests. The default for this is `Authorization`. - **Allow Query Parameter As Well As Header**: Set this option to enable checking the query parameter as well as the header for an auth token. **This is a setting that might be important if your GQL includes subscription operations**. - **Use Cookie Value**: It is possible to use a cookie value as well as the other two token locations. -- **Enable client certificate**: Select this to use Mutual TLS. See [Mutual TLS]({{< ref "/api-management/client-authentication#use-mutual-tls" >}}) for details on implementing mutual TLS. +- **Enable client certificate**: Select this to use Mutual TLS. See [Mutual TLS]({{< ref "api-management/client-authentication#use-mutual-tls" >}}) for details on implementing mutual TLS. ### Step 5: Save the API @@ -262,7 +262,7 @@ We just sent an API definition to the Tyk `/apis` endpoint. API definitions are Notice that when creating a GQL API you need to include your GQL service schema in the API definition. Tyk Gateway doesn't have the capacity to introspect your GQL service on its own. -Including the correct schema allows Tyk Gateway to validate incoming requests against it. More on validation can be found [here]({{< ref "/graphql/validation">}}) +Including the correct schema allows Tyk Gateway to validate incoming requests against it. More on validation can be found [here]({{< ref "graphql/validation">}}) {{< /note >}} @@ -280,7 +280,7 @@ Your GQL API is now ready to use. We recommend that you secure any GQL API that {{< tabs_end >}} Check the following docs for more on GraphQL-specific security options: -* [Field based permissions]({{< ref "/graphql/field-based-permissions">}}) -* [Complexity limiting]({{< ref "/graphql/complexity-limiting">}}) -* [Introspection]({{< ref "/graphql/introspection">}}) +* [Field based permissions]({{< ref "graphql/field-based-permissions">}}) +* [Complexity limiting]({{< ref "graphql/complexity-limiting">}}) +* [Introspection]({{< ref "graphql/introspection">}}) diff --git a/tyk-docs/content/graphql/gql-headers.md b/tyk-docs/content/graphql/gql-headers.md index 28c6bea7b6..d318767954 100644 --- a/tyk-docs/content/graphql/gql-headers.md +++ b/tyk-docs/content/graphql/gql-headers.md @@ -46,7 +46,7 @@ It is possible to enrich any GQL request proxied through Tyk Gateway with additi {{< img src="/img/dashboard/graphql/headers-gql-request.png" alt="Request headers" >}} -**Request headers** values can be defined as context variables. To know how to refer to request context variables check [this page]({{< ref "/context-variables">}}). +**Request headers** values can be defined as context variables. To know how to refer to request context variables check [this page]({{< ref "context-variables">}}). Any header key/value pair defined in **Request headers** will only be used to inject headers into requests proxied through the Gateway. It will not be used to introspect the upstream schema from Tyk Dashboard. diff --git a/tyk-docs/content/graphql/graphql-playground.md b/tyk-docs/content/graphql/graphql-playground.md index cd3a023bec..2f24d3295f 100644 --- a/tyk-docs/content/graphql/graphql-playground.md +++ b/tyk-docs/content/graphql/graphql-playground.md @@ -143,7 +143,7 @@ Debugging a GraphQL API might require additional headers to be passed to the req {{< img src="/img/dashboard/udg/getting-started/headers.png" alt="Headers" >}} -You can also [forward headers]({{< ref "graphql/gql-headers.md" >}}) from your client request to the upstream data sources. +You can also [forward headers]({{< ref "graphql/gql-headers" >}}) from your client request to the upstream data sources. ### Logs diff --git a/tyk-docs/content/graphql/graphql-websockets.md b/tyk-docs/content/graphql/graphql-websockets.md index 59e60b6977..a7b969f4d9 100644 --- a/tyk-docs/content/graphql/graphql-websockets.md +++ b/tyk-docs/content/graphql/graphql-websockets.md @@ -131,4 +131,4 @@ For Subscriptions, the Tyk Gateway will respond with a stream of `data` messages ### Upstream connections -For setting up upstream connections (between Tyk Gateway and Upstream) please refer to the [GraphQL Subscriptions Key Concept]({{< ref "/getting-started/key-concepts/graphql-subscriptions" >}}). +For setting up upstream connections (between Tyk Gateway and Upstream) please refer to the [GraphQL Subscriptions Key Concept]({{< ref "getting-started/key-concepts/graphql-subscriptions" >}}). diff --git a/tyk-docs/content/graphql/introspection.md b/tyk-docs/content/graphql/introspection.md index 6510f0d37b..6925516bcf 100644 --- a/tyk-docs/content/graphql/introspection.md +++ b/tyk-docs/content/graphql/introspection.md @@ -13,7 +13,7 @@ A GraphQL server can provide information about its schema. This functionality is If **introspection** is a completely new concept for you, browse through the official [GraphQL Specification](https://spec.graphql.org/October2021/#sec-Introspection) published by the GrapQL Foundation to find out more. -When [creating a GraphQL proxy]({{< ref "/graphql/creating-gql-api">}}) in Tyk Dashboard an introspection query is used to fetch the schema from the GraphQL upstream and display it in the schema tab. +When [creating a GraphQL proxy]({{< ref "graphql/creating-gql-api">}}) in Tyk Dashboard an introspection query is used to fetch the schema from the GraphQL upstream and display it in the schema tab. {{< note success >}} **Note** @@ -31,7 +31,7 @@ In the *Create new API* screen you have to tick the **Upstream Protected** optio - From the **Upstream protected by** section choose the right option for your case: Headers or Certificate. - Choosing **Headers** will allow you to add multiple key/value pairs in *Introsopection headers* section. - - You can also **Persist headers for future use** by ticking that option. This will save information you provided in case in the future your schema changes and you need to sync it again. To understand better where this information will be saved, go to [GQL Headers]({{< ref "/graphql/gql-headers">}}). To read more about schema syncing go [here]({{< ref "/graphql/syncing-schema">}}). + - You can also **Persist headers for future use** by ticking that option. This will save information you provided in case in the future your schema changes and you need to sync it again. To understand better where this information will be saved, go to [GQL Headers]({{< ref "graphql/gql-headers">}}). To read more about schema syncing go [here]({{< ref "graphql/syncing-schema">}}). - Choosing **Certificate** will allow you to provide *Domain* details and either *Select certificate* or *Enter certificate ID*. ## Turning off introspection @@ -65,7 +65,7 @@ Because introspection control in Tyk works on Policy and Key level, it means you {{< tab_end >}} {{< tab_start "Tyk APIs" >}} -First, you need to learn [how to create a security policy with Tyk API]({{< ref "getting-started/create-security-policy" >}}) or [how to create an API Key with Tyk API]({{< ref "/api-management/policies#access-key-level-security" >}}). +First, you need to learn [how to create a security policy with Tyk API]({{< ref "getting-started/create-security-policy" >}}) or [how to create an API Key with Tyk API]({{< ref "api-management/policies#access-key-level-security" >}}). Once you learn how to utilize the API to create a security policy or a key, you can use the following snippet: diff --git a/tyk-docs/content/graphql/introspection/introspection-queries.md b/tyk-docs/content/graphql/introspection/introspection-queries.md index 49e5558c8a..da18a92d62 100644 --- a/tyk-docs/content/graphql/introspection/introspection-queries.md +++ b/tyk-docs/content/graphql/introspection/introspection-queries.md @@ -307,4 +307,4 @@ If you prefer to introspect GraphQL all at once, you can do that by sending this ``` -Tyk also allows you to block introspection queries for security reasons if you wish to do so. More information on how to do that is provided [here]({{< ref "/graphql/introspection#turning-off-introspection">}}). \ No newline at end of file +Tyk also allows you to block introspection queries for security reasons if you wish to do so. More information on how to do that is provided [here]({{< ref "graphql/introspection#turning-off-introspection">}}). \ No newline at end of file diff --git a/tyk-docs/content/graphql/persisted-queries.md b/tyk-docs/content/graphql/persisted-queries.md index 27070a02ad..68f19f5f55 100644 --- a/tyk-docs/content/graphql/persisted-queries.md +++ b/tyk-docs/content/graphql/persisted-queries.md @@ -121,7 +121,7 @@ If you run a request to your proxy, you should get a response similar to this: ### Dynamic variables -We have seen support for passing static variable values via the API definition, but there will be cases where we want to extract variables from the request header or URL. More information about available request context variables in Tyk can be found [here]({{(< ref "/context-variables">)}}) +We have seen support for passing static variable values via the API definition, but there will be cases where we want to extract variables from the request header or URL. More information about available request context variables in Tyk can be found [here]({{(< ref "context-variables">)}}) Below is an examples of using an incoming `code` header value as a variable in `persist_graphql` middleware configuration: diff --git a/tyk-docs/content/graphql/syncing-schema.md b/tyk-docs/content/graphql/syncing-schema.md index 6e3360d1aa..36142c7619 100644 --- a/tyk-docs/content/graphql/syncing-schema.md +++ b/tyk-docs/content/graphql/syncing-schema.md @@ -23,4 +23,4 @@ Syncing schemas is only available for proxy-only GraphQL APIs and **not** for UD {{< img src="/img/dashboard/graphql/schema_sync.png" alt="Sync Schema Button" >}} - If your upstream is protected then you need to make sure you provide Tyk with the authorization details to execute introspection query correctly. You can add those detail while [creating GQL API]({{< ref "/graphql/introspection#introspection-for-protected-upstreams">}}) or using [Introspection headers]({{< ref "/graphql/gql-headers#introspection-headers">}}) later on. \ No newline at end of file + If your upstream is protected then you need to make sure you provide Tyk with the authorization details to execute introspection query correctly. You can add those detail while [creating GQL API]({{< ref "graphql/introspection#introspection-for-protected-upstreams">}}) or using [Introspection headers]({{< ref "graphql/gql-headers#introspection-headers">}}) later on. \ No newline at end of file diff --git a/tyk-docs/content/key-concepts/grpc-proxy.md b/tyk-docs/content/key-concepts/grpc-proxy.md index f6f409389d..532c697cfc 100644 --- a/tyk-docs/content/key-concepts/grpc-proxy.md +++ b/tyk-docs/content/key-concepts/grpc-proxy.md @@ -34,17 +34,17 @@ For scenarios where you want to connect two services calling each other or just Tyk supports all kinds of gRPC streaming (client streaming, server streaming and bidirectional streaming). It requires you to set a low value for `flush_interval`, this is required in order to forward data to the downstream target as soon as the upstream target replies. A high flush interval will delay this communication. We recommend the lowest possible value: 1 (1 millisecond). You set this value in your `tyk.conf` file in the `http_server_options.flush_interval` option. ### Mutual Authentication -Tyk supports Mutual Authentication in gRPC. See [Mutual TLS]({{< ref "/api-management/client-authentication#use-mutual-tls" >}}) to configure Mutual Authentication in Tyk. +Tyk supports Mutual Authentication in gRPC. See [Mutual TLS]({{< ref "api-management/client-authentication#use-mutual-tls" >}}) to configure Mutual Authentication in Tyk. ### Basic Authentication -Tyk supports Basic Authentication in gRPC. See [Basic Authentication]({{< ref "/api-management/client-authentication#use-basic-authentication" >}}) to configure Basic Authentication in Tyk. +Tyk supports Basic Authentication in gRPC. See [Basic Authentication]({{< ref "api-management/client-authentication#use-basic-authentication" >}}) to configure Basic Authentication in Tyk. After setting your Tyk configuration, all you need to do is to send credentials with the correct base64 format in an `Authorization` header from your gRPC client. `Basic base64Encode(username:password)` ### Token Based Authentication -Tyk supports Token Based Authentication in gRPC. See [Bearer Tokens]({{< ref "/api-management/client-authentication#use-auth-tokens" >}}) to configure Token Based Authentication in Tyk. +Tyk supports Token Based Authentication in gRPC. See [Bearer Tokens]({{< ref "api-management/client-authentication#use-auth-tokens" >}}) to configure Token Based Authentication in Tyk. After setting your Tyk configuration, all you need to do is to send a token in an `Authorization` header from your gRPC client. diff --git a/tyk-docs/content/portal/api-consumer.md b/tyk-docs/content/portal/api-consumer.md new file mode 100644 index 0000000000..335c637fc0 --- /dev/null +++ b/tyk-docs/content/portal/api-consumer.md @@ -0,0 +1,380 @@ +--- +title: "Developers / API Consumers" +date: 2022-02-10 +linkTitle: API Management +tags: ["Developer Portal", "Tyk", "API Consumer", "Developer", "Organization", "Invite Codes", "Consumer Registration"] +keywords: ["Developer Portal", "Tyk", "API Consumer", "Developer", "Organization", "Invite Codes", "Consumer Registration"] +description: "How to configure API Consumers in Tyk developer portal" +aliases: + - /tyk-stack/tyk-developer-portal/enterprise-developer-portal/managing-access/manage-api-consumers + - /tyk-stack/tyk-developer-portal/enterprise-developer-portal/managing-access/manage-api-consumer-organisations + - /tyk-developer-portal/tyk-enterprise-developer-portal/api-consumer-portal/reset-password + - /tyk-developer-portal/tyk-enterprise-developer-portal/api-consumer-portal/access-api-product + - /tyk-developer-portal/tyk-enterprise-developer-portal/api-consumer-portal/register-portal + - /tyk-stack/tyk-developer-portal/enterprise-developer-portal/managing-access/add-organisations + - /tyk-stack/tyk-developer-portal/enterprise-developer-portal/managing-access/approve-self-registering-requests + - /tyk-stack/tyk-developer-portal/enterprise-developer-portal/managing-access/invite-codes + - /tyk-stack/tyk-developer-portal/enterprise-developer-portal/managing-access/manage-api-users + - /tyk-developer-portal/tyk-enterprise-developer-portal/api-consumer-portal + - /tyk-stack/tyk-developer-portal/enterprise-developer-portal/managing-access/managing-access +--- + +{{< note success >}} +**Tyk Enterprise Developer Portal** + +If you are interested in getting access contact us at [support@tyk.io]() + +{{< /note >}} + +## Manage API Consumers + +External developers are referred to as API Consumers. In the Tyk Developer portal, you can manage external API Consumers via the admin dashboard. + +### Glossary + +**API Consumers** - Refers to the whole section within the admin dashboard that manages individual external users, teams, and organizations. + +**Organizations** - An organization can represent larger business units of a company. It works as a container for various teams and users. An organization can be used which can include multiple teams. + +**Teams** - Teams are used to bundle multiple users, a team always needs to be part of an organization. + +**Users** - External developers / portal users. A user can belong to multiple teams but can only belong to one organization. + +{{< img src="/img/dashboard/portal-management/enterprise-portal/api-consumers-menu.png" alt="Portal API Consumers menu" >}} + +### How does the API Consumer section work? + +When installing the Tyk Portal, by default the API Consumers section will already have a default organization with a default team added. This means, if your specific use case doesn't require multiple organizations and teams, you can get started straight away and invite a new external user to the developer portal, adding them to the default organization and default team. + +If your use case requires adding a new organization, see [step by step guide]({{< ref "portal/api-consumer#manage-api-consumer-organizations" >}}). When an organization is created, a default team tied to that organization is automatically generated too. Teams and organizations can have different visibility when it comes to API Products and plans. This can be managed within the [catalog section]({{< ref "portal/api-provider#create-a-new-catalog" >}}). + +## Manage API Consumer Individual Users + +### Register a New API User + +Developers need to register with the portal to access API Products and manage their applications. This section outlines how developers can register with the portal, access API Products, and reset their passwords if needed. + +There are two ways for registering a new account +1. Self signup via the form. +2. Receive an invitation from the portal admin. + +Here you’ll learn about how to add and invite a new external user to the developer portal. + +**Prerequisites** + +- A Tyk portal installation +- Log in to the portal admin app + +#### Self Registration/Signup + +To use the self sign up flow, you’ll need to: +1. Access the Portal and click **REGISTER**. + + {{< img src="/img/dashboard/portal-management/enterprise-portal/portal-login.png" alt="Portal login and Register menu" >}} + +2. Complete the **Create an Account** form. + + {{< img src="/img/dashboard/portal-management/enterprise-portal/create-account.png" alt="Form to create a developer portal account" >}} + +3. Click **Register to developer portal**. +4. If the portal allows signup without approval, you'll get a message that allows you to log in straight away. + + {{< img src="/img/dashboard/portal-management/enterprise-portal/account-registered.png" alt="Account registered to allow immediate access to the portal" >}} + +5. If the portal requires an admin to approve a registration request, after submitting the **Create an Account** form, you will get the following message. + + {{< img src="/img/dashboard/portal-management/enterprise-portal/account-email-popup.png" alt="Registration account submitted for admin approval" >}} + +#### Invite a New User + +1. From the **API Consumers > Users** menu Click **Add new user**. + + {{< img src="/img/dashboard/portal-management/enterprise-portal/users-menu.png" alt="Portal API Users menu" >}} + +2. In the **Add user** dialog, enter **First** and **Last** names, and **Email**. +3. Select an organization to which to register your user. +4. You can also set a password for a user by typing it in the **Set password** field. Check the **User must change password at the next login** if you wish your developer to change their password at next login. + + Please note, that you can either send the invite email or set the password yourself, but you cannot use both methods. + + {{< img src="/img/dashboard/portal-management/enterprise-portal/add-users.png" alt="Add API Users dialog" width="600">}} + +5. Click **Save** to add your user. +6. To generate the invite email, click **More Options** in the Overview section and then **Send invite**. + + The user will receive an email with a link to the registration form. This option is only available if you didn't set the password before. + To customize the invite email, please refer to the [Email customization section]({{< ref "portal/customization#configure-email-notifications" >}}) for guidance. + + {{< img src="/img/dashboard/portal-management/enterprise-portal/users-send-invite.png" alt="Users Send invite dialog" >}} + +#### Invite Codes + +Here you’ll learn about how to create invite codes to add a new external user to the developer portal. Invite codes can be used to simplify user onboarding. Using invite codes, users will be directly assigned to a team and organization, giving them the same access rights as this team. For example you can use invite codes to: +- Run a promotional campaign like a Hackathon and give access to specific plans to the users. +- Onboard a partner company into the portal by giving them this code for anyone registering. + +**Prerequisites** + +- A Tyk Enterprise portal installation +- A portal admin app login + +**Step by step instructions** + +1. From the **API Consumers > Invite Codes** menu, click **Add**. + + {{< img src="/img/dashboard/portal-management/enterprise-portal/invite-codes.png" alt="Invite Codes menu" >}} + {{< img src="/img/dashboard/portal-management/enterprise-portal/add-invite-code.png" alt="Invite Codes dialog" >}} + +2. Add the form details: + + - Set your desired **Quota**. Quota is the max number of slots available for your invite code. E.g. if set 100, this code can be used by the top 100 users. + - Set an expiry date, this is the date on which the code will expire and no more users can sign up to it. + - Set state to **Active** - this means the code is activated and developers can start using it. + - Specify the team that any new users that register with this invite code will be added to. + + {{< img src="/img/dashboard/portal-management/enterprise-portal/invite-code-dialog.png" alt="Invite Codes dialog" >}} + +3. **Save** the invite code. +4. Share the invite codes. When the saving changes a new Invite code was created and can be viewed in the overview table. To share the invite code, copy it and send to your developer teams/users. + + {{< img src="/img/dashboard/portal-management/enterprise-portal/share-invite-codes.png" alt="Share Invite Codes dialog" >}} + +### Approve Self Registering Requests + +#### Manual Approval + +This section explains how to approve/reject external users self-registering requests to the developer portal. Follow the step-by-step guide. + +**Prerequisites** + +A Tyk Enterprise portal installation + +**Step by step instructions** + +1. Click *Users* from the **API Consumers** menu + +{{< img src="/img/dashboard/portal-management/enterprise-portal/users-menu.png" alt="Portal API Users menu" >}} + +2. When a new user has self-registered to access the developer portal, their user profile will be added to the overview in the **Users** section. + +{{< img src="/img/dashboard/portal-management/enterprise-portal/approve-users1.png" alt="List of Users for your portal app" >}} + +3. To approve a user, click on an **inactive** user. Select **Activate developer** from the dialog. + +{{< img src="/img/dashboard/portal-management/enterprise-portal/activate-user.png" alt="Select Activate developer" >}} + +#### Automatically Approve User Registrations + +If you want all Users to be automatically approved this setting can be changed under **Settings > General**. Select **Auto approve developer regestering requests**. + +{{< img src="/img/dashboard/portal-management/enterprise-portal/auto-approve-users.png" alt="Setting to automatically approve user registrations" >}} + + +## Manage API Consumer Organizations + +Quite often, API Providers have to provide API Products to other companies. In fact, 90% of our customers say that their primary audience is other companies. In this case, they are dealing with not just individual developers but with teams of developers. +Unlike individual developers, companies require more sophisticated machinery to access API credentials: +* Usually, a company is represented by a team of developers, not just an individual. Communication between API Providers and API Consumers mustn’t rely on a single individual that may leave a company or be fired; +* API Consumers need to share access credentials securely within their team. Without that capability, they have to share credentials with internal communication tools, which is a horrible practice. Credentials may be stolen, exposed to an incorrect audience, or not appropriately updated; +* Those teams have an internal hierarchy: some users have admin responsibilities with broader permissions, while other teammates’ permissions are restricted to only accessing API Credentials; +* API Consumers should be able to maintain their teams by themselves: invite new members or remove ones that left the team. + +So, simply put, there are two main challenges that the new API Consumer organization management capability solves: +* How to share securely share access credentials between team members; +* How to manage user permissions on the API consumer side. + +**Prerequisites** + +Before starting, you need to set up an email server because it’s used to send invitations to API Consumer team members. +Please refer to the email notifications documentation to set up the email server. + +Please refer to the [email notification section]({{< ref "portal/settings#email-configuration" >}}) for further instructions for setting up the email server. + +### Admin Settings and Governance + +You can control if API Consumers can register an organization and if such registration requires approval from the portal admins. +To enable API Consumer organization registration, navigate to the Settings/General menu and scroll to the API Consumer access section. In that section, there are two settings that control API Consumer registration: +* **Enable API consumers to register organizations**: when this setting is enabled, API Consumers can register organizations, and the respective button appears in the navigation menu; +* **Auto-approve API consumers registering organization**: When this setting is enabled, no approval is required from the portal admins for an API Consumer to register an organization. If this setting is disabled, API Consumer can register organizations, but they won’t be able to invite team members. + +
This is how it looks in the portal's UI: +{{< img src="/img/dashboard/portal-management/enterprise-portal/api_consumer_org_registration_settings.png" alt="Organization registration settings" >}} + +
To proceed with the following steps, enable the Enable API consumers setting to register organizations. + +### Self Registration + +**Steps for Configuration** + +1. **Request org registration** + + Register a developer account or use an existing one and log in to the developer portal as a developer. + To start the organization registration flow, click on the **Create an organization** button in the top right corner of the screen. + {{< img src="/img/dashboard/portal-management/enterprise-portal/become_an_organisation_navbar.png" alt="Become an organization button" >}} + +

You will be navigated to the screen where you can specify the name of your future organization. + {{< img src="/img/dashboard/portal-management/enterprise-portal/specify_name_of_an_organisation.png" alt="Specify name of the organization" >}} + +

If the **Auto-approve API consumers registering organization** setting is enabled, the new organization will instantly be provisioned. + {{< img src="/img/dashboard/portal-management/enterprise-portal/org_registration_is_approved.png" alt="Organization registration is approved" >}} + +

Otherwise, the developer will have to wait for approval from admin users. + {{< img src="/img/dashboard/portal-management/enterprise-portal/org_registration_is_pending.png" alt="Organization registration is pending" >}} + +2. **Approve or reject organization registration requests** + + If the **Auto-approve API consumers registering organization** setting is disabled and the email settings are configured correctly, the admin users will be notified about the new organization registration request via email. + {{< img src="/img/dashboard/portal-management/enterprise-portal/new_org_request_email.png" alt="New organization registration request notification" >}} + +

If the **Auto-approve API consumers registering organization** setting is disabled, the new API Consumer organizations won’t be immediately provisioned. + As an admin user, you can approve or reject organization registration requests from the Organization menu. + {{< img src="/img/dashboard/portal-management/enterprise-portal/pending_org_registration_admin.png" alt="New organization registration request view" >}} + + When admin users approve or reject organization registration requests, the respective email notification is sent to API Consumers. + + Notification when organization request is approved: + {{< img src="/img/dashboard/portal-management/enterprise-portal/org_request_approved_email.png" alt="Organization registration request is approved" >}} + +

Notification when organization request is rejected: + {{< img src="/img/dashboard/portal-management/enterprise-portal/org_request_rejected_email.png" alt="Organization registration request is rejected" >}} + +

Both emails are customizable. Refer to [the email customization documentation]({{< ref "portal/customization#configure-email-notifications" >}}) for further information on the email customization. + +3. **Invite or remove teammates** + + Once admin users approve the organization registration request, API Consumers can invite teammates. + As an API Consumer, navigate to the Dashboard to invite new teammates. + {{< img src="/img/dashboard/portal-management/enterprise-portal/navigate_to_dashboard.png" alt="Navigate to the dashboard" >}} + +

Then select the Users tab in the side menu. + {{< img src="/img/dashboard/portal-management/enterprise-portal/open_users_tab.png" alt="Navigate to the Users tab" >}} + +

You can add a new team member to your API Consumer organization in the Users tab. To invite a new team member, specify their first and last name, email address, and role. + {{< img src="/img/dashboard/portal-management/enterprise-portal/invite_team_member.png" alt="Invite new team member" >}} + +

There are two possible roles for API Consumers: + * Super admin; + * Team member. + + The difference between these two roles is that the Super admins can invite or remove users from their organization and manage applications, while the Team members can only manage applications. + +

Once the invitation is sent, the invited team member should receive the following email: + {{< img src="/img/dashboard/portal-management/enterprise-portal/team-member-invitation-email.png" alt="Invite new team member email" >}} + +

The invited team member can use the link from the email to register in the portal and join the organization. + {{< img src="/img/dashboard/portal-management/enterprise-portal/register-new-user.png" alt="Invite new team member email" >}} + +4. **Manage API Consumers' role** + + API Consumer Super admins can manage users in their organizations. To do so, navigate to the Users menu in the Dashboard and select a user to edit. + {{< img src="/img/dashboard/portal-management/enterprise-portal/edit_api_consumer.png" alt="Edit API Consumer profile" >}} + +

As a Super admin, you can change users’ first and last names and roles. The changes will take effect immediately. + {{< img src="/img/dashboard/portal-management/enterprise-portal/manage_api_consumer_profile.png" alt="Manage API Consumer profile" >}} + +5. **Sharing assets between teammates** + + Now, when any team member creates an application, all other team members can access it and use the credentials. + {{< img src="/img/dashboard/portal-management/enterprise-portal/share_credentials_between_api_consumers.png" alt="Share credentials between API Consumers" >}} + +### Manually Create Organizations + +In this section, you’ll learn how to create a new organization for your external API Consumers. + +**Prerequisites** + +- A Tyk Enterprise portal installation +- A portal admin app login + +**Step by step instructions** + +1. From the **API Consumers > Organizations** menu, click **Add**. + + {{< img src="/img/dashboard/portal-management/enterprise-portal/organisations-menu.png" alt="Portal Organizations menu" >}} + {{< img src="/img/dashboard/portal-management/enterprise-portal/add-org2.png" alt="Add a new Organization" >}} + +2. Enter the name of your new organization + + {{< img src="/img/dashboard/portal-management/enterprise-portal/add-orgs.png" alt="Add a new Organization" >}} + +3. Click **Save** to create your new organization. A new default-team will also automatically created that is tied to your new organization. + +{{< note success >}} +**Note** + +If you want to edit the default team name you can do so by navigating to **Teams**, open up the team associated with the organization you created and edit the name as required. + +{{< /note >}} + + +## Access an API Product + +This section explains how to access API products as a registered portal developer + +**Prerequisites** + +You need to have successfully registered for a portal. + +**Step by step instructions** + +1. Login to the external developer portal +2. Go to **Catalogs** to view the available API products + +{{< note success >}} +**Note** + +Using the filter at the top, you can filter on the different created catalogs including different API Products, e.g. a custom catalog that only you and a specific team can access. + +{{< /note >}} + +3. When selecting on a product, you can click **More info** to access the product overview page. +4. On the product overview page, you can view documentation for each API included in the API product. You can also view information about which catalog the API product is part of. Each catalog may have different plans available so you need to select a catalog based on which plan you want to access. +5. Click **Add to cart**. + +{{< note success >}} +**Note** + +You can add multiple products to the cart. You can receive one set of access credentials for multiple products as long as they are part of the same cart. If you are adding two products to the cart, and they are part of different catalogs, e.g. Private and Public, you will need to go through two request flows, and you will get two different sets of credentials for each API Product. + +{{< /note >}} + +6. Add the details needed and select a subscription plan for your API Product(s) chosen. +7. Create a new app or add to an existing one. If you already have an existing app created you can access it via the drop down or select **Create a new app** to add the credentials to an existing app. +8. Click **Request access**. +9. Navigate to My apps and view the app you created. Depending if the plan requires manual approval by an admin or not, you will either see that the request is pending or you can see the approved access credentials immediately you can start using them. + +{{< note success >}} +**Note** + +When sending a query, make sure to use the Base URL outlined in the overview of the API Product. + +{{< /note >}} + +## Reset Password + +This page goes through the reset password routine. If you have forgotten your password you can easily reset it via the portal. + +1. Navigate to the live portal and click **Login** and you’ll get to the login screen. + + {{< img src="/img/dashboard/portal-management/enterprise-portal/login-portal-forgot.png" alt="Portal login screen" >}} + +2. At the login screen click **Forgot Password?** link and you’ll be redirected to the reset password form. + + {{< img src="/img/dashboard/portal-management/enterprise-portal/forgot-password.png" alt="Forgot Password email address" >}} + +3. Enter your email address, and click **Reset** and you’ll see this message. + + {{< img src="/img/dashboard/portal-management/enterprise-portal/forgot-password-email.png" alt="Forgot Password email sent" >}} + +4. Check your email and you should have received an email that contains a link the following format: +`https://the-portal-domain.com/auth/reset/code?token=` + +5. Click on the link and you will be taken to the reset password screen. +6. Enter your new password and click **Reset**. + + {{< img src="/img/dashboard/portal-management/enterprise-portal/reset-password-request.png" alt="Enter and confirm new password" >}} + +7. Click **Login again** to go to the Login screen. + + {{< img src="/img/dashboard/portal-management/enterprise-portal/reset-done.png" alt="Enter and confirm new password" >}} + diff --git a/tyk-docs/content/tyk-stack/tyk-developer-portal/enterprise-developer-portal/api-access/dynamic-client-registration.md b/tyk-docs/content/portal/api-provider.md similarity index 55% rename from tyk-docs/content/tyk-stack/tyk-developer-portal/enterprise-developer-portal/api-access/dynamic-client-registration.md rename to tyk-docs/content/portal/api-provider.md index f1e7288bbc..f9fa93a76e 100644 --- a/tyk-docs/content/tyk-stack/tyk-developer-portal/enterprise-developer-portal/api-access/dynamic-client-registration.md +++ b/tyk-docs/content/portal/api-provider.md @@ -1,12 +1,18 @@ --- -title: "Dynamic Client Registration" -date: 2022-02-11 -tags: ["Tyk Developer Portal","Enterprise Portal", "Dynamic client registration", "DCR", "Okta", "Keycloak"] -description: "How to configure the Dynamic client registration flow with the Enterprise Portal" -menu: - main: - parent: "API Access" -weight: 5 +title: "API Providers" +date: 2025-02-10 +linkTitle: API Management +tags: ["Developer Portal", "Tyk", "Managing Access", "Catalogs", "Rate Limit", "Dynamic Client Registration", "Documenting APIs"] +keywords: ["Developer Portal", "Tyk", "Managing Access", "Catalogs", "Rate Limit", "Dynamic Client Registration", "Documenting APIs"] +description: "How to configure API Providers in Tyk developer portal" +aliases: + - /tyk-stack/tyk-developer-portal/enterprise-developer-portal/api-access/dynamic-client-registration + - /tyk-stack/tyk-developer-portal/enterprise-developer-portal/managing-access/manage-catalogues + - /tyk-stack/tyk-developer-portal/enterprise-developer-portal/api-access/api-access + - /tyk-stack/tyk-developer-portal/enterprise-developer-portal/api-access/approve-requests + - /tyk-stack/tyk-developer-portal/enterprise-developer-portal/api-access/manage-apps-credentials + - /tyk-stack/tyk-developer-portal/enterprise-developer-portal/api-access/configuring-custom-rate-limit-keys + - /tyk-stack/tyk-developer-portal/enterprise-developer-portal/getting-started-with-enterprise-portal/manage-get-started-guides-for-api-products --- {{< note success >}} @@ -16,7 +22,187 @@ If you are interested in getting access contact us at [support@tyk.io](}} -## Introduction +## How to Manage API Access? + +The Tyk Enterprise Developer portal provides a special container for access credentials - an application. Applications hold developer credentials that can be used to access APIs published on the portal with a specific plan. A sample relationship between applications, credentials, API Products, and plans is demonstrated in the diagram below. + +{{< img src="/img/dashboard/portal-management/enterprise-portal/portal-apps-structure.png" alt="Sample application setup" >}} + +Credentials are generated by the Tyk Dashboard when an admin user approves a provisioning request or when provisioning requests are configured to be approved automatically. The provisioning flow is demonstrated in the diagram below. + +{{< img src="/img/dashboard/portal-management/enterprise-portal/portal-provisioning-flow.png" alt="Portal provisioning flow" >}} + +This section describes how admin users can manage applications and configure the settings of provisioning requests. + +## Provisioning Request + +### Approve or Reject Provisioning Request + +When an external developer is looking to access a specific API(s) as a part of an API Product, they will request access via the public facing portal. + +**Prerequisites** + +- A Tyk Enterprise portal installation +- A portal admin app login +- Log in to a provisioning request sent by an external API consumer + +**Step by step instructions** + +1. Log in to the portal admin app +2. Navigate to **Provisioning Requests** +3. Select which request you want to approve +4. Click the **more** symbol and select either **approve** or **reject** + +{{< img src="/img/dashboard/portal-management/enterprise-portal/approve-request.png" alt="Approve or reject an API provisioning request" >}} + +### Configure Auto Approval + +You can auto approve provisioning requests. From the **Plans** section of the admin app, edit a plan and select **Auto-approve provisioning request** from the **Plan Settings**. By default this setting is not selected, requiring manual approval of each request. Click **Save Changes** to enable this setting. + +{{< img src="/img/dashboard/portal-management/enterprise-portal/auto-approve-requests.png" alt="Auto Approve API provisioning requests" >}} + +## Manage Apps and Credentials + +**Prerequisites** + +- A Tyk Enterprise portal installation +- A portal admin app login +- A login to the already created app by an external API consumer + +### To view existing apps + +1. In the portal admin app, go to **Apps**. If there are some apps already created by external API-consumers, you’ll see them in the overview list. +{{< img src="/img/dashboard/portal-management/enterprise-portal/list-of-app-admin-app.png" alt="List of applications" >}} + +2. Click on an app from the overview page. +3. On the right hand slideout, you will see the information about the created app. + +### Revoke app credentials + +1. In the portal admin app, go to **Apps** and open the app. +2. Click **credentials**. This means the developer will no longer be able to access the API Product via the selected plan. These actions will have an immediate effect. + +## Manage Catalogs + +Catalogs are a way for you to tailor the audience for API products and Plans. You can, for example create a Partner Catalog, with a specific API product tailored to them and a preferential plan not available in your public portal. + +In this section, you will learn about how catalogs work and how to create a new catalog to expose your API products and plans. + +**Prerequisites** + +- Connect to a provider [Tyk Self-Managed]({{< ref "portal/overview#connect-to-a-provider" >}}) +- Create [policies with enforced access rights]({{< ref "portal/overview#create-api-products-and-plans" >}}) (API Product in the Portal) +- Create one or more [policies with enforced rate limit and quotas]({{< ref "portal/overview#create-api-products-and-plans" >}}) (Plan in the Portal) + +### Create a New Catalog + +1. Navigate to the **Catalog section** section + + {{< img src="/img/dashboard/portal-management/enterprise-portal/catalogue-menu.png" alt="Catalogue menu" >}} + +2. Click Create a new catalog + + {{< img src="/img/dashboard/portal-management/enterprise-portal/portal-managing-access-create-catalogue.png" alt="Create a new catalogue" >}} + +3. Enter Name and Path URL + + {{< img src="/img/dashboard/portal-management/enterprise-portal/portal-managing-access-add-name.png" alt="Name the new catalogue" >}} + +4. Set the access required for the catalog e.g. Public, Private or Custom + + - Public: External developers can access the catalog + - Private: The catalog is only visible to developers that are logged in + - Custom: Only selected teams can access this catalog + +5. [If creating a custom catalog] Under Audience, select one or multiple teams that you want to have access to this catalog. + +{{< note success >}} +**Note** + +For this audience to apply, the visibility needs to be set to custom. + +{{< /note >}} + +6. Select the catalog content in terms of which API Products and plans this catalog should contain. + + + +## Configure Rate Limits + +Different business models may require applying rate limits and quotas not only by credentials but also by other entities, e.g. per application, per developer, per organization etc. +For example, if an API Product is sold to a B2B customer, the quota of API calls is usually applied to all developers and their respective applications combined, in addition to a specific credential. + +To enable this, Tyk introduced support for custom rate limit keys in [Tyk 5.3.0]({{< ref "developer-support/release-notes/dashboard#530-release-notes" >}}). This guide explains how to configure custom rate limit keys. + +**Prerequisites** + +This capability works with [Tyk 5.3.0]({{< ref "developer-support/release-notes/dashboard#530-release-notes" >}}) or higher. + +### Configuring custom rate limit keys for policies in Tyk Dashboard + +{{< note >}} +**Note** + +If you are using Tyk Developer Portal version 1.13.0 or later, you can configure the custom rate limit keys directly from the Developer Portal in the Advanced settings (optional) colapsible section of the Plan's view (by Credentials metadata). +{{< img src="img/dashboard/portal-management/enterprise-portal/portal-plan-advanced-settings.png" alt="Add Plan Advanced Settings" >}} +{{< /note >}} + +Custom rate limit keys are applied at a policy level. When a custom rate limit key is specified, quota, rate limit and throttling will be calculated against the specified value and not against a credential ID. + +To specify a custom rate limit key, add to a policy a new metadata field called `rate_limit_pattern`. +In the value field you can specify any value or expression that you want to use as a custom rate limit key for your APIs. +The `rate_limit_pattern` field supports referencing session metadata using `$tyk_meta.FIELD_NAME` syntax. +In addition, it's possible to concatenate multiple values together using the pipe operator (`|`). + +For instance, if you want to specify a rate limit pattern to calculate the rate limit for a combination of developers and plans, where all credentials of a developer using the same plan share the same rate limit, you can use the following expression. +This assumes that the `DeveloperID` and `PlanID` metadata fields are available in a session: + +```gotemplate +$tyk_meta.DeveloperID|$tyk_meta.PlanID +``` + +Here's how it looks like in the Tyk Dashboard UI: + +{{< img src="/img/dashboard/portal-management/enterprise-portal/configuring-custom-rate-limit-keys.png" alt="Configuring custom rate limit keys" >}} + +
+ +{{< note success >}} +**Updating credential metadata** + +Please note that the custom rate limit key capability uses only metadata objects, such as credentials metadata available in a session. +Therefore, if the `rate_limit_pattern` relies on credentials metadata, this capability will work only if those values are present. +If, after evaluating the `rate_limit_pattern`, its value is equal to an empty string, the rate limiter behavior defaults to rate limiting by credential IDs. + +{{< /note >}} + +### Using custom rate limit keys with the portal + +The Tyk Enterprise Developer Portal facilitates the configuration of various rate limiting options based on a business model for API Products published in the portal. + +To achieve this, the portal, by default, populates the following attributes in the credential metadata, which can be used as part of a custom rate limit key: +- **ApplicationID**: The ID of the application to which the credential belongs. +- **DeveloperID**: The ID of the developer who created the credential. +- **OrganizationID**: The ID of the organization to which the developer belongs. + +Additionally, it's possible to attach [custom attribute values]({{< ref "portal/customization#add-custom-attributes-to-the-user-model" >}}) defined in a developer profile as metadata fields to credentials. + +When a credential is provisioned by the portal, all the fields described above are added as metadata values to the credential, making them valid options for configuring the rate limit key: + +{{< img src="/img/dashboard/portal-management/enterprise-portal/credential-metadata.png" alt="Credential's metadata" >}} + +This approach allows the portal to seamlessly apply rate limits based on any combination of the aforementioned fields and other custom metadata objects defined in policies used for plans or products. This is in addition to credentials. + +--- + +{{< note success >}} +**Tyk Enterprise Developer Portal** + +If you are interested in getting access contact us at [support@tyk.io]() + +{{< /note >}} + +## Dynamic Client Registration **Why OAuth2.0 is important** @@ -37,10 +223,10 @@ Tyk Enterprise Developer portal allows API providers to set up a connection with
-## Prerequisites for getting started +### Prerequisites Before getting starting with configuring the portal, it's required to configure your Identity provider and the Dashboard beforehand. -### Create an initial access token +#### Create an initial access token Before setting up Tyk Enterprise Developer Portal to work with DCR, you need to configure the identity provider. Please refer to the guides for popular providers to create the initial access token for DCR: * [Gluu](https://gluu.org/docs/gluu-server/4.0/admin-guide/openid-connect/#dynamic-client-registration) * [Curity](https://curity.io/docs/idsvr/latest/token-service-admin-guide/dcr.html) @@ -53,7 +239,7 @@ Before setting up Tyk Enterprise Developer Portal to work with DCR, you need to Whilst many providers require initial access tokens, they are optional. Please refer to your provider documentation to confirm if required. {{< /note >}} -### Create OAuth2.0 scopes to enforce access control and rate limit +#### Create OAuth2.0 scopes to enforce access control and rate limit Tyk uses OAuth2.0 scope to enforce access control and rate limit for API Products. Therefore, creating at least two scopes for an API Product and plan is required. @@ -124,7 +310,7 @@ Navigate to the Tyk Dashboard and create two policies: one for a plan and one fo {{< img src="/img/dashboard/portal-management/enterprise-portal/create-jwt-policy-for-plan.png" alt="Create a policy for a plan" >}} -### Create the No Operation policy and API +#### Create the No Operation policy and API {{< note >}} **Note** @@ -133,7 +319,7 @@ You can skip this step if you are using Tyk Developer Portal version 1.13.0 or l Go directly to [Configure Tyk Enterprise Developer Portal to work with an identity provider](#configure-tyk-enterprise-developer-portal-to-work-with-an-identity-provider). {{< /note >}} -Tyk requires any API that uses the scope to policy mapping to have [a default policy]({{< ref "/api-management/client-authentication#use-json-web-tokens-jwt" >}} ). Access rights and rate limits defined in the default policy take priority over other policies, including policies for the API Product and plan. +Tyk requires any API that uses the scope to policy mapping to have [a default policy]({{< ref "api-management/client-authentication#use-json-web-tokens-jwt" >}} ). Access rights and rate limits defined in the default policy take priority over other policies, including policies for the API Product and plan. To avoid that, you need to create the No Operation API and policy that won't grant access to the APIs included in the API Product but will satisfy the requirement for a default policy. @@ -163,7 +349,7 @@ To avoid that, you need to create the No Operation API and policy that won't gra Configure the No Operation policy and save it: {{< img src="/img/dashboard/portal-management/enterprise-portal/save-the-noop-policy.png" alt="Save the No Operation policy" >}} -### Configure scope to policy mapping +#### Configure scope to policy mapping {{< note >}} **Note** @@ -200,12 +386,12 @@ To achieve that, perform the following steps for each API included in the API Pr {{< img src="/img/dashboard/portal-management/enterprise-portal/add-a-scope-to-policy-mapping-for-the-plan-scope.png" alt="Add scope to policy mapping for the plan scope" >}} -## Configure Tyk Enterprise Developer Portal to work with an identity provider +### Configure Tyk Enterprise Developer Portal to work with an identity provider Set up the portal to work with your IdP. -### Configure the App registration settings +#### Configure the App registration settings In the portal, navigate to the `OAuth2.0 Providers` menu section. In that section, you need to configure the connection settings to the IdP and define one or more types (configurations) of OAuth 2.0 clients. For instance, you can define two types of OAuth 2.0 clients: * A confidential client that supports the Client credential grant type for backend integrations; @@ -214,7 +400,7 @@ In the portal, navigate to the `OAuth2.0 Providers` menu section. In that sectio Each configuration of OAuth 2.0 client could be associated with one or multiple API Products so that when an API Consumer requests access to an API Product, they can select a client type that is more suitable for their use case. -#### Specify connection setting to your IdP +##### Specify connection setting to your IdP To connect the portal to the IdP, you need to specify the following settings: * OIDC well-known configuration URL. @@ -222,7 +408,7 @@ To connect the portal to the IdP, you need to specify the following settings: First of all, select your IdP from the `Identity provider` dropdown list. Different IdPs have slightly different approaches to DCR implementation, so the portal will use a driver that is specific to your IdP. If your IdP is not present in the dropdown list, select the `Other` option. In that case, the portal will use the most standard implementation of the DCR driver, which implements the DCR flow as defined in the RFC. -Then you need to specify the connection settings: [the initial access token and the well-known endpoint]({{< ref "tyk-stack/tyk-developer-portal/enterprise-developer-portal/api-access/dynamic-client-registration#create-an-initial-access-token" >}}). If your Identity Provider uses certificates that are not trusted, the portal will not work with it by default. To bypass certificate verification, you can select the `SSL secure skip verify` checkbox. +Then you need to specify the connection settings: [the initial access token and the well-known endpoint]({{< ref "portal/api-provider#create-an-initial-access-token" >}}). If your Identity Provider uses certificates that are not trusted, the portal will not work with it by default. To bypass certificate verification, you can select the `SSL secure skip verify` checkbox. The below example demonstrates how to achieve that with Keycloak and Okta in the tabs below. @@ -246,14 +432,14 @@ The below example demonstrates how to achieve that with Keycloak and Okta in the {{< tabs_end >}} -#### Create client configurations +##### Create client configurations Once the connection settings are specified, you need to create one or multiple types of clients. You might have multiple types of clients that are suitable for different use cases, such as backend integration or web applications. You need at least one type of client for the DCR flow to work. To add the first client type, scroll down to the `Client Types` section and click on the `Add client type` button. To configure a client type, you need to specify the following settings: * **Client type display name.** This name will be displayed to API consumers when they check out API products. Try to make it descriptive and short, so it's easier for API consumers to understand. -* **Description.** A more verbose description of a client type can be provided in this field. By default, we do not display this on the checkout page, but you can customize the respective template and make the description visible to API consumers. Please refer to [the customization section]({{< ref "tyk-stack/tyk-developer-portal/enterprise-developer-portal/customise-enterprise-portal/full-customisation/full-customisation" >}}) for guidance. +* **Description.** A more verbose description of a client type can be provided in this field. By default, we do not display this on the checkout page, but you can customize the respective template and make the description visible to API consumers. Please refer to [the customization section]({{< ref "portal/customization#" >}}) for guidance. * **Allowed response_types.** Response types associated with this type of client as per [the OIDC spec](https://openid.net/specs/openid-connect-core-1_0-17.html). * **Allowed grant_types.** Grant types that this type of client will support as per [the OIDC spec](https://openid.net/specs/openid-connect-core-1_0-17.html). * **Token endpoint auth methods.** The token endpoint that will be used by this type of client as per [the OIDC spec](https://openid.net/specs/openid-connect-core-1_0-17.html). @@ -293,9 +479,9 @@ To configure API Products to work with the DCR flow, you need to: For achieving this, navigate to the `API Products` menu and select the particular API product you want to use for the DCR flow. Next, go to the ‘App registration configs’ section and enable the ‘Enable dynamic client registration’ checkbox. -After that, specify the scope for this API product. You should have at least one scope that was created in [the Prerequisites for getting started]({{< ref "tyk-stack/tyk-developer-portal/enterprise-developer-portal/api-access/dynamic-client-registration#prerequisites-for-getting-started" >}}). If you need to specify more than one scope, you can separate them with spaces. +After that, specify the scope for this API product. You should have at least one scope that was created in [the Prerequisites for getting started]({{< ref "portal/api-provider#prerequisites" >}}). If you need to specify more than one scope, you can separate them with spaces. -Finally, select one or multiple types of clients that were created in [the Create client configurations]({{< ref "tyk-stack/tyk-developer-portal/enterprise-developer-portal/api-access/dynamic-client-registration#create-client-configurations" >}}) section of this guide to associate them with that product. +Finally, select one or multiple types of clients that were created in [the Create client configurations]({{< ref "portal/api-provider#create-client-configurations" >}}) section of this guide to associate them with that product. {{< tabs_start >}} @@ -323,9 +509,9 @@ From version 1.13.0, you can complete the DCR configuration for a product under {{< img src="img/dashboard/portal-management/enterprise-portal/portal-product-dcr.png" alt="Add DCR settings" >}} {{< /note >}} -#### Configure plans for the DCR flow +##### Configure plans for the DCR flow -The last step is to configure the plans you want to use with the DCR flow. To do this, go to the portal's `Plans` menu section and specify the OAuth2.0 scope to use with each plan. You should have at least one scope that was created in [the Prerequisites for getting started]({{< ref "tyk-stack/tyk-developer-portal/enterprise-developer-portal/api-access/dynamic-client-registration#prerequisites-for-getting-started" >}}). If you need to specify more than one scope, you can separate them with spaces. +The last step is to configure the plans you want to use with the DCR flow. To do this, go to the portal's `Plans` menu section and specify the OAuth2.0 scope to use with each plan. You should have at least one scope that was created in [the Prerequisites for getting started]({{< ref "portal/api-provider#prerequisites" >}}). If you need to specify more than one scope, you can separate them with spaces. {{< img src="/img/dashboard/portal-management/enterprise-portal/configure-plan-for-the-dcr-flow.png" alt="Configure a plan to work with the DCR flow" >}}
@@ -337,15 +523,14 @@ From version 1.13.0, you can complete the DCR configuration for a plan under the {{< img src="img/dashboard/portal-management/enterprise-portal/portal-plan-advanced-settings.png" alt="Add Plan Advanced Settings" >}} {{< /note >}} -## Test the DCR flow +### Test the DCR flow To test the DCR flow, you need to perform the following actions: - Request access to the API product and plan you have selected for the DCR flow as a developer. - Approve the access request as an admin. - As a developer, copy the access credentials and obtain an access token. - As a developer, make an API call to verify the flow's functionality. - -### Request access to the API Product +#### Request access to the API Product To request access to the DCR enabled API Product: - Log in as a developer and navigate to the catalog page. - Select the DCR enabled API Product and add it to the shopping cart. @@ -355,12 +540,12 @@ To request access to the DCR enabled API Product: - Finally, select the applicable type of client and click on the `Submit request` button. {{< img src="/img/dashboard/portal-management/enterprise-portal/request-access-to-the-dcr-enabled-product.png" alt="Request access to the DCR enabled product" width="600" >}} -### Approve the access request +#### Approve the access request To approve the access request, navigate to the `Access requests` menu in the portal, select the access request and approve it by clicking on the `Approve` button. {{< img src="/img/dashboard/portal-management/enterprise-portal/approve-dcr-access-request.png" alt="Approve DCR access request" >}} -### Obtain an access token -Once the access request is approved, the developer should receive an email informing them of the approval. Please refer to [the email customization section]({{< ref "tyk-stack/tyk-developer-portal/enterprise-developer-portal/customise-enterprise-portal/full-customisation/email-customization.md" >}}) if you wish to change the email template. +#### Obtain an access token +Once the access request is approved, the developer should receive an email informing them of the approval. Please refer to [the email customization section]({{< ref "portal/customization#configure-email-notifications" >}}) if you wish to change the email template. As a developer, navigate to the `My Dashboard` section in the developer portal, select the application, and copy the OAuth 2.0 credentials. @@ -379,7 +564,7 @@ Since in this example we use the client_secret_basic token endpoint authenticati As a result, you should receive a JWT access token containing the required scopes: {{< img src="/img/dashboard/portal-management/enterprise-portal/jwt.png" alt="An example of a JWT" width="600" >}} -### Make an API Call +#### Make an API Call Finally, use the access token to make an API call and test the flow functionality: ```curl curl --location --request GET 'http://localhost:8080/payment-api/get' \ @@ -401,3 +586,101 @@ You should receive the following response: "url": "http://httpbin.org/get" } ``` + +## Documentation for your API Products + +As an API Provider, you can package APIs into API Products and supply them with documentation to deliver value to your internal and external API Consumers.
+API documentation is essential for API Products. Good API documentation goes beyond just Open API Specification. To make your API Products easy to use and engaging, you can add technical and business guides to the documentation of your API Products, for instance: +* Get started guides explaining how to start with your API Product; +* Use-cases; +* Business documentation that explains the business value and helps to convince decision-makers. + +Tyk Enterprise Developer portal provides two ways of documenting APIs: +* Open API Specifications for APIs; +* The Get started documentation feature enables API Providers to add as many technical and business guides to API Products as needed. + +This section explains how to add the Get started documentation to API Products. + +### Create Get started guides for your API Product + + +1. **Create and publish an API Product** + + To start with, create and publish an API Product. Please refer to the [Publish API Products and Plans]({{< ref "portal/overview#publish-an-api-product" >}}) page for further guidance. + +2. **Add API Documentation to API Products** + + {{< note >}} +**Note** + +If you are using Tyk Developer Portal version 1.13.0 or later, you can add API Documentation under the `"Getting Started" guides` tab of the API Product's view. +{{< img src="img/dashboard/portal-management/enterprise-portal/portal-product-guides.png" alt="Add Product Guides" >}} + {{< /note >}} + + To add API Documentation, select an API Product and navigate to the API Documentation section. + Click the ‘Add API Documentation’ button to add a new API Documentation page. + {{< img src="img/dashboard/portal-management/enterprise-portal/api-docs-navigate-to-the-api-docs-section.png" alt="Navigate to the API Documentation section" >}} + +
+ + Tyk developer portal stores two versions of each API Documentation page: in Markdown and HTML formats. + You can edit both versions, but only one version of the page will be displayed. + To edit the Markdown version, click on the ‘Change to Markdown’ button. + {{< img src="img/dashboard/portal-management/enterprise-portal/api-documentation-page-change-to-md.png" alt="Toggle the Markdown editor" >}} + +
+ + To edit the HTML version, click the 'Change to HTML' button. + {{< img src="img/dashboard/portal-management/enterprise-portal/api-documentation-page-change-to-html.png" alt="Toggle the HTML editor" >}} + +
+ + To select which version of the documentation to display, check the ‘Display Markdown’ checkbox. + When it’s checked, the portal will display the Markdown version of the page. + Otherwise, portal will display the HTML version. + {{< img src="img/dashboard/portal-management/enterprise-portal/api-documentation-page-display-md-version.png" alt="Display the MD version of the page" >}} + +3. **Add more documentation pages and set the display order** + + With the Tyk developer portal, you can add multiple Markdown and HTML pages to your API Products and select their display order to form a table of contents suitable for your use case. + Please refer to the previous step to add one more documentation page to the API Product. + + To reorder API Documentation items, scroll down to the API Documentation section and click on the ‘Reorder items’ button. + {{< img src="img/dashboard/portal-management/enterprise-portal/api-documentation-page-start-reordering-docs.png" alt="Reorder API Docs" >}} + +
+ + Then click on the ‘Move up’ button to move an API Documentation item one position up, and the ‘Move down’ button to bring it one position down. Click on the 'Done' button and then on the 'Save' button to save your changes. + {{< img src="img/dashboard/portal-management/enterprise-portal/api-documentation-page-move-doc-up.png" alt="Reorder API Docs" >}} + +
+ + Once you save the changes, the new order of items will be reflected in the ‘Get started’ tab. + {{< img src="img/dashboard/portal-management/enterprise-portal/api-documentation-page-get-started-live.png" alt="Reorder API Docs" >}} + +
+ + All documentation pages are listed in the ‘Product docs’ section. You can access and edit the API docs from that section. + {{< img src="img/dashboard/portal-management/enterprise-portal/product-docs-section.png" alt="Product docs section" >}} + +4. **Add tags to blogs and API Products** + + Finally, you can add blog posts to your API Products using the Tags feature. + You can specify tags for API Product and blog posts, then the blog posts that match tags with an API Product will be displayed in the ‘Related blog content’ section. + {{< img src="img/dashboard/portal-management/enterprise-portal/tags-diagram.png" alt="Tags diagram" width="600" height="314" >}} + +
+ + To specify tags for an API Product, select an API Product, scroll down to the ‘Tags’ section, type in a tag, and press the Enter key. + {{< img src="img/dashboard/portal-management/enterprise-portal/tags-section-in-api-product.png" alt="Specify tags for an API Product" >}} + +
+ + To specify tags for a blog post, select a blog post, scroll down to the ‘Tags’ section, type in a tag, and press the Enter key. + {{< img src="img/dashboard/portal-management/enterprise-portal/tags-section-in-blog-post.png" alt="Specify tags for a blog post" >}} + +
+ + The blog posts that match tags with an API Product will be displayed in the ‘Related blog content’ section of the respective API Product page. + {{< img src="img/dashboard/portal-management/enterprise-portal/related-blog-content.png" alt="Related blog content" width="800" height="925">}} + diff --git a/tyk-docs/content/portal/customization.md b/tyk-docs/content/portal/customization.md new file mode 100644 index 0000000000..47830e50ab --- /dev/null +++ b/tyk-docs/content/portal/customization.md @@ -0,0 +1,3106 @@ +--- +title: "Developer Portal Customization" +date: 2025-02-10 +linkTitle: API Management +tags: ["Developer Portal", "Tyk", "Customization", "Webhook", "Email", "Themes", "Templates", "Pages", "Menus", "Branding", "User Model"] +keywords: ["Developer Portal", "Tyk", "Customization", "Webhook", "Email", "Themes", "Templates", "Pages", "Menus", "Branding", "User Model"] +description: "Customization options for enterprise developer portal" +aliases: + - /product-stack/tyk-enterprise-developer-portal/portal-customisation/configure-webhooks + - /product-stack/tyk-enterprise-developer-portal/portal-customisation/customise-user-model + - /tyk-stack/tyk-developer-portal/enterprise-developer-portal/customise-enterprise-portal/quick-customisation + - /tyk-stack/tyk-developer-portal/enterprise-developer-portal/customise-enterprise-portal/full-customisation/full-customisation + - /tyk-stack/tyk-developer-portal/enterprise-developer-portal/customise-enterprise-portal/full-customisation/templates + - /product-stack/tyk-enterprise-developer-portal/upgrading/theme-upgrades + - /tyk-stack/tyk-developer-portal/enterprise-developer-portal/customise-enterprise-portal/full-customisation/edit-manage-page-content + - /tyk-stack/tyk-developer-portal/enterprise-developer-portal/customise-enterprise-portal/full-customisation/menus-customisation + - /tyk-stack/tyk-developer-portal/enterprise-developer-portal/customise-enterprise-portal/full-customisation/file-structure-concepts + - /tyk-stack/tyk-developer-portal/enterprise-developer-portal/customise-enterprise-portal/full-customisation/create-new-page-template + - /tyk-stack/tyk-developer-portal/enterprise-developer-portal/customise-enterprise-portal/full-customisation/email-customization + - /product-stack/tyk-enterprise-developer-portal/getting-started/customize-sign-up-form + - /product-stack/tyk-enterprise-developer-portal/getting-started/customize-products-and-plans +--- + +{{< note success >}} +**Tyk Enterprise Developer Portal** + +If you are interested in getting access contact us at [support@tyk.io]() + +{{< /note >}} + +## Portal Customization Overview + +The Tyk Enterprise Developer Portal uses themes for customizing the live portal. We provide an out of the box theme that is using our own branding, it’s called the `default` theme. You are welcome to use it and modify it for your needs, yet if you want to start with a blank page, you can also create a completely new theme. + +This section provides a complete guide to full customization from the developer's point of view. + +## Configure Themes + +### What is a Theme? + +The Tyk Enterprise Developer Portal uses **themes** for customizing the live portal. We provide an out of the box theme that is using our own branding, it’s called the `default` theme. You are welcome to use it and modify it for your needs, yet if you want to start with a blank page, you can also create a completely new theme. + +The following section explains how they are structured and their main concepts. We recommend you to read this if you are creating your own theme, or making extensive changes to the ones we provide. + +### File Structure of a Theme + +Generally speaking, a theme defines an application’s styling, templates and scripts. +In the Tyk Developer Portal a `themes` folder is located in the root of the application and is the directory where each theme folder must be added. If you navigate to `path /themes/` you’ll see our default theme which has the following structure: + +{{< img src="/img/dashboard/portal-management/enterprise-portal/theme-file-structure.png" alt="Default Tyk Enterprise Portal theme structure" >}} + +- Manifest file (`theme.json`): It uses JSON syntax to define theme metadata (name, version and author) as well as a list of templates that are part of the theme. +- `assets`: It intended for static assets like CSS, JS or images that are used by the theme. All contents from this directory are mounted under the `/assets` path in the portal HTTP server. +- `layouts`: The layout is the top level view of your theme. +- `views`: The view is rendered as a part of a layout. Each view can be rendered using a different layout. +- `partials`: Partials provide an easier way to handle snippets of code that are reused across different views or layouts, for example if you want to inject a JS snippet that’s used in different places, you could set this code in a partial and include it anywhere by using the appropriate 'Go template directive'. In this way you could improve code readability and organize the theme in the most efficient way. + +#### Manifest File + +This file should sit in the root of a theme and hold the theme configuration. You can define a name and your templates along other options such as the version and the author. + +You can find an example of the manifest within the `default` theme that is located in `/themes/default`. The syntax looks as follows: + +```json +{ + "name": "default", + "version": "0.0.1", + "author": "Tyk Technologies Ltd. ", + "templates": [ + { + "name": "Content Page", + "template": "page", + "layout": "site_layout" + }, + { + "name": "Portal Home", + "template": "portal_home", + "layout": "portal_layout" + }, + { + "name": "Home", + "template": "home", + "layout": "portal_layout" + }, + { + "name": "Catalogue", + "template": "catalogue", + "layout": "portal_layout" + } + ] +} +``` + +The `templates` field establishes a list of available templates. Every template consists of three fields where `name` is a user-friendly name that will be seen on the Admin app when creating a page. `template` is a reference to the template file itself. `layout` is a reference to the layout that will be used to render the previously set template. + +To illustrate the current template hierarchy, this is what a typically rendered page would look like. The `layout` would be the top level template and base structure of the page: +{{< img src="/img/dashboard/portal-management/enterprise-portal/portal-template-layout.png" alt="Template structure" >}} + + +Also note that the Developer Portal will let you use not just multiple `layouts` and `views` but also any combination of them. These combinations are set in your manifest file (`theme.json`). + +Regarding `partials`, even though the illustration above shows two partials embedded on the `view` section, `partials` are intended for using in any place. You should be able to embed a `partial` directly into a layout, or even in multiple layouts. + +Content blocks are explored more deeply in the next sections. To summarise, its relationship with the above hierarchy is when rendering a particular page, a `layout`, a `view` and potentially a combination of partials get loaded from the theme directory. Content blocks are different because their content gets dynamically populated by database content. These contents are created from the Admin section. + +To be Concluded: + +- A layout is the wrapper of everything you want to include inside it. So, typically it would consist of tags such as ``, ``, ``, ``, and `<body>`. +- A `template` is what we would inject in a layout and specifically within the `<body>` of a layout. +- A `partial` can be, for example, the navigation menu so that you can inject it in the layout and it will be visible every time this layout is used + +#### Go Templates + +All theme template files use the Go template syntax. You will find every file in the layouts and views. Partials directory uses the `.tmpl` file extension, which is the default Go template extension. Go templates work in a similar way to ERB or EJS templates by letting the user mix HTML code with dynamic values. Sample syntax is as follows: + +`{{ render “top_nav” }}` + +The code sample above would execute the `render` template helper, which is a common function that’s used to inject code from other `views` into the current one. You may use this to embed content from other parts of the theme, typically `partials` or `views`. In this case, it will insert a `view` or `partial` named `top_nav` to the template where it’s used. + +The same delimiters `{{` and `}}` are used for all Go template directives. We’ll explore some of them in the upcoming sections. + +See the [Go package template documentation](https://pkg.go.dev/text/template#pkg-overview) for more information. + +#### Content Blocks + +The Developer Portal themes use content blocks to facilitate content management. A content block is defined as a part of a `view` by using a particular template directive in combination with a name or ID to identify the given block. For example, if you check the `home` template in the default theme (`themes/default/views/home.tmpl`), you will find the following code: + +```go +div class="container"> + <div class="row"> + <div class="col-sm-6"> + <div class="text-container"> + <h1>{{.page.Title}}</h1> + <p>{{.blocks.HeaderDescription.Content}}</p> + <a href="{{.blocks.HeaderButtonLink.Content}}" class="btn btn-primary">{{.blocks.HeaderButtonLabel.Content}}</a> + </div> +…. +``` + +There are four code references in the above snippet. In this example we have a header, some text and then a button that act as a link. Let's see what each one is and how it correlates with the UI. + +1. `{{ .page.Title }}`. This is the `Title` input in the form UI (Screenshot #1) +1. `{{ .blocks.HeaderDescription.Content }}`. This is the `HeaderDescription` input in the form UI (Screenshot #2) +2. `{{ .blocks.HeaderButtonLink.Content }}`. This is the `HeaderDescription` input in the form UI (Screenshot #3) +3. `{{ .blocks.HeaderButtonLabel.Content }}`. This is the `HeaderButtonLabel` input in the form UI (Screenshot #4) + +{{< img src="/img/dashboard/portal-management/enterprise-portal/go-template-ui.png" alt="Go template blocks and portal UI" >}} + +This will display in your portal as following: + +{{< img src="/img/dashboard/portal-management/enterprise-portal/example-portal-content-block.png" alt="Example Portal content block" >}} + +In order for a page to render properly the content manager will need to be aware of the content blocks that are required by a particular template. + +### Managing Themes + +The Tyk Enterprise Developer Portal enables the admin users and developers to manage themes and select which theme is visible in the portal. +To enable this capability, the portal has theme management UI. + +#### Create a Theme +Follow the example below to create a new theme called "TestTheme" using the default theme as a blueprint: + +1. As an admin user, navigate to the Theme management UI and download the default theme. The Tyk Enterprise Developer Portal doesn't allow modifications to the default theme so that you will always have access to the vanilla theme. + {{< img src="/img/dashboard/portal-management/enterprise-portal/download-default-theme.png" alt="Download default theme" >}} +2. Unzip the theme and rename it by modifying the Manifest file as described above. + {{< img src="/img/dashboard/portal-management/enterprise-portal/rename-a-theme.png" alt="Rename theme" >}} +3. You can also modify other assets in the theme as described later in this guide. Once all modifications are done, you need to zip the theme and upload it to the portal. + {{< img src="/img/dashboard/portal-management/enterprise-portal/compress-a-theme.png" alt="Zip theme" >}} +4. To upload the theme as an admin user, navigate to **Themes** and click on the **Add new theme** button. Please note that the size of individual files should not exceed 5 MB and the total size of all files in the theme should not exceed `PORTAL_MAX_UPLOAD_SIZE`. This parameter is [configurable]({{< ref "product-stack/tyk-enterprise-developer-portal/deploy/configuration#portal_max_upload_size" >}}). + {{< img src="/img/dashboard/portal-management/enterprise-portal/add-a-new-theme.png" alt="Add new theme" >}} +5. Then click on the **Add theme file** button. + {{< img src="/img/dashboard/portal-management/enterprise-portal/add-theme-file.png" alt="Add theme file" >}} +6. Select the archive that you've created earlier and click on the **Save** button. + {{< img src="/img/dashboard/portal-management/enterprise-portal/save-new-theme.png" alt="Save new theme" >}} +7. Now you should see a success message meaning the theme was successfully created. + {{< img src="/img/dashboard/portal-management/enterprise-portal/new-theme-is-created.png" alt="Theme is created" >}} + +#### Preview a Theme +The Tyk Enterprise Developer Portal enables the admin users to preview the theme before it gets reflected on the public-facing portal. This enables to review the changes that are made to the theme before exposing them to the developer community. +1. To preview a theme as an admin user, navigate to the **Themes** menu. Select a theme, and click on the **Preview** button. + {{< img src="/img/dashboard/portal-management/enterprise-portal/preview-theme-button.png" alt="Preview theme" >}} +2. The previewer will open the selected theme in a new tab. Now you can browse your theme and review the changes. For the demonstration purposes, we've modified the API Catalog page so it displays "Modified catalog" instead of "Product Catalogs". + {{< img src="/img/dashboard/portal-management/enterprise-portal/theme-preview.png" alt="Preview theme" >}} +3. Once the review is done, you can quit the preview by clicking on the **Quit preview button**. + {{< img src="/img/dashboard/portal-management/enterprise-portal/quit-theme-preview.png" alt="Quite theme preview" >}} + +#### Activate a Theme +The Tyk Enterprise Developer Portal enables you to have multiple themes at the same time but only one of them is active. +1. As an admin user, navigate to the **Themes** menu. The current status of each theme is displayed in the **Status** column. + {{< img src="/img/dashboard/portal-management/enterprise-portal/default-theme-is-current.png" alt="Default theme is the current theme" >}} +2. To activate the new theme, click on the **Activate** button. + {{< img src="/img/dashboard/portal-management/enterprise-portal/activate-a-theme.png" alt="Activate theme" >}} +3. The selected theme is now active and displayed to all API consumers on the Live portal. + {{< img src="/img/dashboard/portal-management/enterprise-portal/modified-theme-is-active.png" alt="Modified theme is activated" >}} + +#### Modify an Existing Theme +The Tyk Enterprise Developer Portal enables modification to any existing theme, except the default one. +1. To start modification of any existing theme, navigate to the **Themes** menu and download the theme package. + {{< img src="/img/dashboard/portal-management/enterprise-portal/download-a-theme.png" alt="Download existing theme" >}} +2. Unzip the package, do any required modification and zip it back. You should keep the name of the theme. If you need to change the name of the theme, you will need to create a new theme as described above. +3. As an admin user, navigate to the **Themes** menu and select the modified theme. + {{< img src="/img/dashboard/portal-management/enterprise-portal/select-a-modified-theme.png" alt="Select modified theme" >}} +4. Click on the **Add Theme File** button and select the theme archive. + {{< img src="/img/dashboard/portal-management/enterprise-portal/add-theme-file-to-existing-theme.png" alt="Add theme file" >}} +5. Click on the **Save changes** button to save changes to the theme. + {{< img src="/img/dashboard/portal-management/enterprise-portal/save-changes-to-theme.png" alt="Save changes" >}} +6. If the theme is the current changes to the Live portal, it will be applied immediately. Otherwise, you can preview and activate the theme as described above. + +### Upgrading Themes + +The Tyk Enterprise Developer Portal does not automatically update the default theme with each new release of the product, because doing so could result in the loss of customizations made by customers. +Therefore, customers are required to manually upgrade their themes to access the latest updates and fixes. We recommend using GitFlow for the latest theme updates. + +Alternatively, you can download the theme package from the **Releases** section of the [portal-default-theme](https://github.com/TykTechnologies/portal-default-theme) repository. +However, we advise against this method, as it requires you to merge your customized theme with the downloaded one, which is much simpler to accomplish via GitFlow. +Follow the guide below to obtain the latest version of the portal theme and merge it with your customized version. + +#### Merge Latest Changes using Gitflow + +The default theme for the Tyk Enterprise Developer Portal is located in the [portal-default-theme](https://github.com/TykTechnologies/portal-default-theme) repository. +The `main` branch contains code corresponding to the latest stable release. If you wish to check out a specific release (e.g., v1.8.3), you can use tags: + +```console +git checkout tags/1.8.3 -b my-custom-theme branch +``` + +To organize local development in a way that facilitates seamless theme updates using git merge, follow the steps below: +1. Fork the `portal-default-theme` repo in [github](https://github.com/TykTechnologies/portal-default-theme). + {{< img src="/img/dashboard/portal-management/enterprise-portal/fork-portal-theme-repo.png" alt="Fork the portal-theme repo" >}} + +2. Clone the forked repository: +```console +git clone https://github.com/my-github-profile/portal-default-theme && cd ./portal-default-theme +``` + +3. If you have an existing repository, check if you already have the `portal-default-theme` repo among your remotes before adding it. Execute the following command to check: +```console +git remote -v | grep 'TykTechnologies/portal-default-theme' +``` + +Skip the next step if you see the following: +```console +# portal-default-theme https://github.com/TykTechnologies/portal-default-theme (fetch) +# portal-default-theme https://github.com/TykTechnologies/portal-default-theme (push) +``` + +If the output of the above command is empty, proceed to step 5 to add the `portal-default-theme`. + +4. Add the `portal-default-theme` to the remotes by executing the following command: +```console +git remote add portal-default-theme https://github.com/TykTechnologies/portal-default-theme +``` + +5. Create a new local branch that tracks the remote `main` branch. That branch will mirror the latest changes from the `portal-default-theme` main. You will be using it to import the latest changes from the `portal-default-theme` to your custom theme: +```console +git fetch portal-default-theme main:portal-default-theme-main +``` + +If you have an existing local branch that tracks the `main` branch in the `portal-default-theme` repo, you can just pull the latest updates: +```console +git checkout portal-default-theme-main +git pull portal-default-theme main +``` + +6. To start customizing the theme, create a local develop branch from the `portal-default-theme-main`: +```console +git checkout portal-default-theme-main +git checkout -b dev-branch +``` + +7. Once the required customizations are completed, commit the changes to the `dev-branch`. + +8. Merge the latest updates from the `portal-default-theme` into the `dev-branch`. Please remember to always pull the latest changes from the `portal-default-theme-main` branch before merging: + - Checkout to the portal-default-theme-main and fetch the latest changes. + ```console + git checkout portal-default-theme-main + git pull portal-default-theme main + ``` + - Checkout the dev-branch and merge the changes from the portal-default-theme-main to retrieve the latest changes from the portal-default-theme repo with the customized theme. + ```console + git checkout dev-branch + git merge portal-default-theme-main + ``` + +Finally, address merge conflicts and commit changes. + +{{< note success >}} +**You have successfully updated your custom theme** + +Now you can repeat the above described process when upgrading the portal version to make sure you have incorporated the latest theme changes to your customized theme. + +{{< /note >}} + +#### Upload the Theme to the Portal +Once you have merged your local changes with the latest changes from the `portal-default-theme` repo, you need to create a zip archive with the customized theme and upload it to the portal. +1. Create a zip archive with the customized theme. Make sure you zip the content of the `src` folder and not the folder itself. To create a zip archive with the customized theme execute the following: + - cd to the `src` directory to make sure you: + ```console + cd ./src + ``` + - zip the content of the `src` directory: + ```console + zip -r9 default.zip + ``` + +2. Upload the theme package that is created in the previous step to the portal. You can use the portal's [Admin dashboard]({{< ref "portal/customization#create-a-theme" >}}) or the [admin API]({{< ref "product-stack/tyk-enterprise-developer-portal/api-documentation/tyk-edp-api" >}}) to do it. +![image](https://github.com/TykTechnologies/tyk-docs/assets/14009/f0e547b2-b521-4c3e-97ce-fd3a2a3b170b) +3. Finally, you need to [activate]({{< ref "portal/customization#activate-a-theme" >}}) the theme so that it will be applied to the portal. + +## Configure Templates + +Templates are a fundamental component of the Tyk Enterprise Developer Portal, enabling dynamic content generation and +customization. The portal uses Golang templates to render the live portal views, allowing you to generate dynamic HTML +by embedding directives inside HTML that are replaced with values when the template is executed. + +Golang's templates use the following syntax: +- `{{ . }}` to output a value +- `{{ .FieldName }}` to access a field of an object +- `{{ .MethodName }}` to call a method on an object +- `{{ if }} {{ else }} {{ end }}` for conditionals +- `{{ range . }} {{ . }} {{ end }}` to iterate over a slice +- Functions can be called like `{{ FuncName . }}` or just `{{ FuncName }}` + +These templates are part of the default theme that ships with the portal, which can be fully customized by modifying the +template files. The templates have access to template data which contains dynamic values that can be rendered into the +HTML. There are also a number of global helper functions available to transform data before output. + +The Tyk Enterprise Developer Portal uses several types of templates to render different parts of the portal: +- Public Pages Templates: Render the portal's publicly accessible pages (such as Home, About Us, and Blog pages), +forming the foundation of your portal's public-facing content. These can be customized through the Pages +[section]({{< ref "portal/customization#edit-page-content" >}}) +of the admin dashboard. +- Private Pages Templates: Responsible for rendering the portal's authenticated user pages, like Profile settings and +My Apps. +- Email Templates: Define the structure and content of emails sent by the portal, such as signup confirmations or access +request approvals. + +Both Public and Private Pages Templates have access to global helper functions (funcmaps) and template-specific data. +Email templates can include template data and specific template functions, but do not have access to the global helper +functions. + +The following sections provide comprehensive information on the various components of the Tyk Enterprise Developer +Portal templates: + +- [Template Data](#template-data): Detailed explanation of the data structures available in different templates. +- [Global Helper Functions](#global-helper-functions): A list of global functions that can be used across templates to +manipulate and display data. +- [Email Templates](#email-templates): Information about email-specific templates and their available data. + +### Template Data + +This section outlines the Tyk Enterprise Developer Portal templates that have access to specific template data. +It's important to note that data availability varies between templates, depending on their context and purpose. +For instance, a product detail template has access to product-specific data that may not be available in a blog listing +template. + +#### Templates with specific template data + +- [Analytics](#analytics) +- [Application Create](#application-create) +- [Application Detail](#application-detail) +- [Blogs](#blogs) +- [Blog Detail](#blog-detail) +- [Cart Checkout](#cart-checkout) +- [Organization User Detail](#organization-user-detail) +- [Organization User Edit](#organization-user-edit) +- [Organization Users List](#organization-users-list) +- [Product Detail](#product-detail) +- [Product OAS Documentation](#product-oas-documentation) + +#### Analytics + +**Template Path**: `themes/default/views/analytics.tmpl` + +This template is used to render the analytics page. + +##### Available Objects + +- `{{ .errors }}`: Map of template errors (Key: category, Value: error message) +- `{{ .apps }}`: List of available applications + +##### Application Attributes + +Accessible via `{{ range .apps }}` + +| Attribute | Description | +|-----------|-------------| +| `{{ .ID }}` | Application ID | +| `{{ .Name }}` | Application name | +| `{{ .Description }}` | Application description | +| `{{ .RedirectURLs }}` | Application redirect URLs | + +###### Example Usage +```html +<select id="analytics-overview-select-apps" class="analytics-select-overview"> + <option value="0" selected>All apps</option> + {{ range $app := .apps }} + <option value="{{ $app.ID }}"> + {{ $app.Name }} + </option> + {{ end }} +</select> +``` + +#### Application Create + +**Template Path**: `themes/default/views/app_form_create.tmpl` + +This template is used to render the application creation form. + +##### Available Objects + +- `{{ .errors }}`: Map of template errors (Key: category, Value: error message) + +###### Example Usage +```html +{{ if .errors }} +{{ range $key, $errs := .errors }} +<div class="alert alert-warning cart-error" role="alert"> + <i class="tyk-icon tykon tykon-warning"></i> + <div class="alert__content"> + <strong>{{ $key }}</strong> + <ul> + {{ range $errs }} + <li>{{ . }}</li> + {{ end }} + </ul> + </div> +</div> +{{ end }} +{{ end }} +``` + +#### Application Detail + +**Template Path**: `themes/default/views/app_form_update.tmpl` + +This template is used to render the application detail and update form. + +##### Available Objects + +- `{{ .errors }}`: Map of template errors (Key: category, Value: error message) +- `{{ .app }}`: Selected application object. +- `{{ .appCerts }}`: Map of application MTLS certificates if applicable (Key: access request ID, Value: certificate) +- `{{ .allCerts }}`: Map of all MTLS certificates stored if applicable (Key: cert fingerprint, Value: cert) + +##### MTLS Certificate Attributes (appCerts) + +| Attribute | Description | +|-----------|-------------| +| `{{ .ID }}` | Certificate ID | +| `{{ .Name }}` | Certificate name | +| `{{ .Fingerprint }}` | Certificate fingerprint | +| `{{ .SignatureAlgorithm }}` | Signature algorithm | +| `{{ .Issuer }}` | Certificate issuer | +| `{{ .IsValid }}` | Boolean indicating if the certificate is valid | +| `{{ .ValidNotBefore }}` | Start date of validity | +| `{{ .ValidNotAfter }}` | End date of validity | +| `{{ .Subject }}` | Certificate subject | + +##### MTLS Certificate Attributes (allCerts) + +| Attribute | Description | +|-----------|-------------| +| `{{ .ID }}` | Certificate ID | +| `{{ .Name }}` | Certificate name | + +##### Client Attributes + +Accessible via `{{ .app }}` + +| Attribute | Description | +|-----------|-------------| +| `{{ .ID }}` | Client ID | +| `{{ .Name }}` | Client name | +| `{{ .Description }}` | Client description | +| `{{ .RedirectURLs }}` | Client redirect URLs | +| `{{ .Credentials }}` | Array of client credentials | +| `{{ .AccessRequests }}` | Array of client access requests | + +##### Client Credentials Attributes + +Accessible via `{{ range $cred := .app.Credentials }}` + +| Attribute | Description | +|-----------|-------------| +| `{{ .ID }}` | Credential ID | +| `{{ .Credential }}` | Credential | +| `{{ .CredentialHash }}` | Credential hash | +| `{{ .OAuthClientID }}` | OAuth client ID | +| `{{ .OAuthClientSecret }}` | OAuth client secret | +| `{{ .Expires }}` | Credential expiration | +| `{{ .AccessRequestID }}` | Access request ID associated with the credential | + +##### Client Access Requests Attributes + +Accessible via `{{ range $acreq := .app.AccessRequests }}` + +| Attribute | Description | +|-----------|-------------| +| `{{ .ID }}` | Access request ID | +| `{{ .Status }}` | Access request status | +| `{{ .UserID }}` | User ID associated with access request | +| `{{ .AuthType }}` | Access request auth type | +| `{{ .DCREnabled }}` | true if access request DCR enabled | +| `{{ .ProvisionImmediately }}` | true if provisioned immediately is enabled | +| `{{ .CatalogueID }}` | Catalogue ID | + +##### Product Attributes (within Access Request) + +Accessible via `{{ $product := $acreq.Product }}` + +| Attribute | Description | +|-----------|-------------| +| `{{ .ID }}` | Product ID | +| `{{ .Name }}` | Product name | +| `{{ .DisplayName }}` | Product display name | +| `{{ .Path }}` | Product path | +| `{{ .Description }}` | Product description | +| `{{ .Content }}` | Product content | +| `{{ .AuthType }}` | Product auth type | +| `{{ .DCREnabled }}` | true if product DCR enabled | + +##### Plan Attributes (within Access Request) + +Accessible via `{{ $acreq.Plan }}` + +| Attribute | Description | +|-----------|-------------| +| `{{ .Name }}` | Plan name | +| `{{ .DisplayName }}` | Plan display name | +| `{{ .Description }}` | Plan description | +| `{{ .AuthType }}` | Plan auth type | +| `{{ .Rate }}` | Plan rate | +| `{{ .Per }}` | Plan period | +| `{{ .QuotaMax }}` | Plan quota maximum | +| `{{ .QuotaRenewalRate }}` | Plan quota renewal rate | +| `{{ .AutoApproveAccessRequests }}` | true if auto-approve access requests is enabled | + +###### Example Usage +```html +<h1>{{ .app.Name >}}</h1> +<p>{{ .app.Description }}</p> +<h2>Credentials</h2> +{{ range $cred := .app.Credentials }} +<div> + <p>ID: {{ $cred.ID }}</p> + <p>OAuth Client ID: {{ $cred.OAuthClientID }}</p> + <p>Expires: {{ $cred.Expires }}</p> +</div> +{{ end }} +<h2>Access Requests</h2> +{{ range $acreq := .app.AccessRequests }} +<div> + <p>ID: {{ $acreq.ID }}</p> + <p>Status: {{ $acreq.Status }}</p> + <p>Product: {{ $acreq.Product.Name }}</p> + <p>Plan: {{ $acreq.Plan.Name }}</p> +</div> +{{ end }} +``` + +#### Blogs + +**Template Path**: `themes/default/views/blog_listing.tmpl` + +This template is used to render the blog listing page. + +##### Available Objects + +- `{{ .posts }}`: List of all published blog posts + +##### Blog Attributes + +Accessible via `{{ range .posts }}` + +| Attribute | Description | +|-----------|-------------| +| `{{ .Title }}` | Blog post title | +| `{{ .Lede }}` | Blog post summary | +| `{{ .Content }}` | Blog post content | +| `{{ .MarkdownContent }}` | Markdown content | +| `{{ .MarkdownEnabled }}` | Boolean for Markdown enablement | +| `{{ .Path }}` | Blog post path | +| `{{ .HeaderImage.URL }}` | Header image URL | +| `{{ .BlogSiteID }}` | Blog site ID | +| `{{ .ProductID }}` | Associated product ID | +| `{{ .AuthorID }}` | Author ID | +| `{{ .URL }}` | Full URL of the blog post | + +###### Example Usage +```html +<h1>Blog Posts</h1> +{{ range .posts }} +<div class="blog-post"> + <h2><a href="{{ .URL }}">{{ .Title }}</a></h2> + <img src="{{ .HeaderImage.URL }}" alt="{{ .Title }}"> + <p>{{ .Lede }}</p> +</div> +{{ end }} +``` + +#### Blog Detail + +**Template Path**: `themes/default/views/blog_detail.tmpl` + +This template is used to render the blog detail page. + +##### Available Objects + +- `{{ .post }}`: The selected blog post object. +- `{{ .latest_posts }}`: List of 3 latest blog posts. + +##### Blog Attributes + +Accessible via `{{ .post }}` or `{{ range .latest_posts }}` + +| Attribute | Description | +|-----------|-------------| +| `{{ .Title }}` | Blog post title | +| `{{ .Lede }}` | Blog post summary | +| `{{ .Content }}` | Blog post content | +| `{{ .MarkdownContent }}` | Markdown content | +| `{{ .MarkdownEnabled }}` | Boolean for Markdown enablement | +| `{{ .Path }}` | Blog post path | +| `{{ .HeaderImage.URL }}` | Header image URL | +| `{{ .BlogSiteID }}` | Blog site ID | +| `{{ .ProductID }}` | Associated product ID | +| `{{ .AuthorID }}` | Author ID | +| `{{ .URL }}` | Full URL of the blog post | + +###### Example Usage +``` +<h1>{{ .post.Title }}</h1> +<img src="{{ .post.HeaderImage.URL }}" alt="{{ .post.Title }}"> +<p>{{ .post.Lede }}</p> +{{ if .post.MarkdownEnabled }} +{{ .post.MarkdownContent | markdownify }} +{{ else }} +{{ .post.Content }} +{{ end }} +<p>Read more at: <a href="{{ .post.URL }}">{{ .post.URL }}</a></p> +<h2>Latest Posts</h2> +{{ range .latest_posts }} +<div> + <h3><a href="{{ .URL }}">{{ .Title }}</a></h3> + <p>{{ .Lede }}</p> +</div> +{{ end }} +``` + +#### Cart Checkout + +**Template Path**: `themes/default/views/portal_checkout.tmpl` + +This template is used to render the cart checkout page. + +##### Available Objects + +- `{{ .cart }}`: Map with the cart items for the current user +- `{{ .apps }}`: List of applications for the current user +- `{{ .catalogue_count }}`: Cart catalogues count +- `{{ .certs }}`: List of MTLS certificates if applicable +- `{{ .errors }}`: Map of template errors (Key: category, Value: error message) +- `{{ .provisioned }}`: Boolean indicating whether an access request has been provisioned for the cart + +##### Application Attributes + +Accessible via `{{ range .apps }}` + +| Attribute | Description | +|-----------|-------------| +| `{{ .Name }}` | Application name | +| `{{ .Description }}` | Application description | +| `{{ .RedirectURLs }}` | Application redirect URLs | + +##### MTLS Certificate Attributes + +Accessible via `{{ range .certs }}` + +| Attribute | Description | +|-----------|-------------| +| `{{ .ID }}` | Certificate ID | +| `{{ .Name }}` | Certificate name | + +##### Cart Item Attributes + +Accessible via `{{ range $key, $value := .cart }}` + +| Attribute | Description | +|-----------|-------------| +| `{{ $value.AuthType }}` | Cart item auth type | +| `{{ $value.Catalogue }}` | Cart item catalogue | +| `{{ $value.Products }}` | Cart item array of products | +| `{{ $value.Plan }}` | Cart item plan | +| `{{ $value.DCREnabled }}` | true if cart order consists of DCR products | + +##### Plan Attributes (Within cart item) + +Accessible via `{{ $plan := $value.Plan }}` + +| Attribute | Description | +|-----------|-------------| +| `{{ .ID }}` | Plan ID | +| `{{ .PlanName }}` | Plan name | +| `{{ .FormatQuota }}` | Formatted quota information | +| `{{ .FormatRateLimit }}` | Formatted rate limit information | + +##### Catalogue Attributes (Within cart item) + +Accessible via `{{ $catalogue := $value.Catalogue }}` + +| Attribute | Description | +|-----------|-------------| +| `{{ .ID }}` | Catalogue ID | + +##### Product Attributes (Within cart item) + +Accessible via `{{ range $product := $value.Products }}` + +| Attribute | Description | +|-----------|-------------| +| `{{ .ID }}` | Product ID | +| `{{ .Name }}` | Product name | +| `{{ .DisplayName }}` | Product display name | +| `{{ .Path }}` | Product path | +| `{{ .Description }}` | Product description | +| `{{ .Content }}` | Product content | +| `{{ .AuthType }}` | Product auth type | +| `{{ .DCREnabled }}` | true if product DCR enabled | +| `{{ .AuthTypes }}` | List of product auth types | + +##### Auth Type Attributes (Within product) + +Accessible via `{{ range $auth_type := $product.AuthTypes }}` + +| Attribute | Description | +|-----------|-------------| +| `{{ .AuthType }}` | Auth type | + +##### DCR Client Template Attributes (Within product) + +Accessible via `{{ range $template := $product.Templates }}` + +| Attribute | Description | +|-----------|-------------| +| `{{ .ID }}` | Template ID | +| `{{ .Name }}` | Template name | +| `{{ .GrantType }}` | Template grant type | +| `{{ .ResponseTypes }}` | Template response types | +| `{{ .TokenEndpointAuthMethod }}` | Template token endpoint auth method | +| `{{ .OktaAppType }}` | Template Okta app type | + +<details> +<summary> <b>Example Usage</b></summary> + +```html +<h1>Cart Checkout</h1> +{{ range $key, $value := .cart }} +<div class="cart-item"> + <p>Auth Type: {{ $value.AuthType }}</p> + <p>Plan: {{ $value.Plan.Name }}</p> + {{ range $product := $value.Products }} + <div class="product"> + <h3>{{ $product.DisplayName }}</h3> + <p>{{ $product.Description }}</p> + <p>Path: {{ $product.Path }}</p> + </div> + {{ end }} +</div> +{{ end }} +<h2>Your Applications</h2> +{{ range $app := .apps }} +<div class="application"> + <h3>{{ $app.Name }}</h3> + <p>{{ $app.Description }}</p> +</div> +{{ end }} +{{ if .certs }} +<h2>MTLS Certificates</h2> +{{ range $cert := .certs }} +<div class="certificate"> + <p>ID: {{ $cert.ID }}</p> + <p>Name: {{ $cert.Name }}</p> +</div> +{{ end }} +{{ end }} +``` +</details> + + +#### Organization User Detail + +**Template Path**: `themes/default/views/user_detail.tmpl` + +This template is used to render the organization user detail page. + +##### Available Objects + +- `{{ .errors }}`: Map of template errors (Key: category, Value: error message) +- `{{ .user }}`: The organization user object. + +##### User Attributes + +Accessible via `{{ .user }}` + +| Attribute/Method | Description | +|-------------------|-------------| +| `{{ .ID }}` | User ID | +| `{{ .First }}` | User name | +| `{{ .Last }}` | User surname | +| `{{ .Email }}` | User email | +| `{{ .OrganisationID }}` | User organization ID | +| `{{ .DisplayName }}` | User complete name | +| `{{ .IdentityProvider }}` | User provider (Portal or Tyk Identity Broker) | +| `{{ .GetOrganisationID }}` | User's organization ID | +| `{{ .IsAdmin }}` | true if user is an admin | +| `{{ .IsOrgAdmin }}` | true if user is an organization admin | +| `{{ .DisplayRole }}` | User's role | + + +<details> +<summary> <b>Example Usage</b></summary> + +```html +<h1>User Details</h1> +{{ if .errors }} +{{ range $key, $errs := .errors }} +<div class="alert alert-warning cart-error error-wrapper" role="alert"> + <i class="tyk-icon tykon tykon-warning"></i> + <div class="alert__content"> + <strong>{{ $key }}</strong> + <ul> + {{ range $errs }} + <li>{{ . }}</li> + {{ end }} + </ul> + </div> +</div> +{{ end }} +{{ end }} +<p>Name: {{ .user.DisplayName }}</p> +<p>Email: {{ .user.Email }}</p> +<p>Role: {{ .user.DisplayRole }}</p> +``` + +</details> + +#### Organization User Edit + +**Template Path**: `themes/default/views/user_edit.tmpl` + +This template is used to render the edit page for organization user. + +##### Available Objects + +- `{{ .errors }}`: Map of template errors (Key: category, Value: error message) +- `{{ .roles }}`: List of possible roles +- `{{ .user }}`: The organization user object. + +##### Role Attributes + +Accessible via `{{ range .roles }}` + +| Attribute | Description | +|-----------|-------------| +| `{{ .ID }}` | Role ID | +| `{{ .DisplayName }}` | Role display name | + +##### User Attributes + +Accessible via `{{ .user }}` + +| Attribute/Method | Description | +|-------------------|-------------| +| `{{ .ID }}` | User ID | +| `{{ .First }}` | User name | +| `{{ .Last }}` | User surname | +| `{{ .Email }}` | User email | +| `{{ .OrganisationID }}` | User organization ID | +| `{{ .DisplayName }}` | User complete name | +| `{{ .IdentityProvider }}` | User provider (Portal or Tyk Identity Broker) | +| `{{ .GetOrganisationID }}` | User's organization ID | +| `{{ .IsAdmin }}` | true if user is an admin | +| `{{ .IsOrgAdmin }}` | true if user is an organization admin | +| `{{ .DisplayRole }}` | User's role | + +<details> +<summary> <b>Example Usage</b></summary> + +```html +<form action="edit" method="post" id="user-edit"> + {{ if .error }} + <div class="alert alert-danger" role="alert"> + {{ .error }} + </div> + {{ end }} + <h2>Developer details</h2> + <div> + <label>Name:</label> + <input type="text" name="first" value="{{ .user.First }}" required /> + </div> + <div> + <label>Last name:</label> + <input type="text" name="last" value="{{ .user.Last }}" required /> + </div> + <div> + <label>Email:</label> + <input type="email" name="email" value="{{ .user.Email }}" required disabled /> + </div> + {{ if .roles }} + <div> + <label>Role:</label> + <select name="role" required> + {{ range $role := .roles }} + <option value="{{ $role.ID }}">{{ $role.DisplayName }}</option> + {{ end }} + </select> + </div> + {{ end }} + <div> + <a href="/portal/private/users">Cancel</a> + <input type="submit" value="Save Changes" /> + </div> +</form> +``` + +</details> + +#### Organization Users List + +**Template Path**: `themes/default/views/user_list.tmpl` + +This template is used to render the list of organization users. + +##### Available Objects + +- `{{ .roles }}`: Map of available roles (Key: role, Value: role display name) + +###### Example Usage +```html +<td> {{ index $roles $userInvite.Role }} </td> +{{ end }} +``` + +#### Product Detail + +**Template Path**: `themes/default/views/portal_product_detail.tmpl` + +This template is used to render the product detail page. + +##### Available Objects + +- `{{ .product }}`: The selected product object +- `{{ .catalogues }}`: List of catalogue objects including the selected product +- `{{ .unique_plans }}`: List of plan objects available for the product +- `{{ .scopes }}`: Product scopes as an array of strings +- `{{ .posts }}`: List of related blog post objects +- `{{ .errors }}`: Map of template errors (Key: category, Value: error message) +- `{{ .added }}`: Boolean indicating if the product is added to the cart + +##### Product Attributes + +Accessible via `{{ .product }}` + +| Attribute | Description | +|-----------|-------------| +| `{{ .ID }}` | Product ID | +| `{{ .Name }}` | Product name | +| `{{ .DisplayName }}` | Product display name | +| `{{ .Path }}` | Product path | +| `{{ .ReferenceID }}` | Product reference ID | +| `{{ .Description }}` | Product description | +| `{{ .AuthType }}` | Product auth type | +| `{{ .Logo.URL }}` | Product logo URL | +| `{{ .Feature }}` | true if the product is featured | +| `{{ .DCREnabled }}` | true if DCR is enabled | +| `{{ .ProviderID }}` | Provider ID | + +##### API Details (Within product) + +Accessible via `{{ .product.APIDetails }}` + +| Attribute | Description | +|-----------|-------------| +| `{{ .Name }}` | API name | +| `{{ .Description }}` | API description | +| `{{ .APIType }}` | API type | +| `{{ .TargetURL }}` | API target URL | +| `{{ .ListenPath }}` | API listen path | +| `{{ .OASUrl }}` | API OAS URL | +| `{{ .Status }}` | "Active" if API status is active, otherwise "Inactive" | + +##### Documentation (Within product) + +Accessible via `{{ .product.Docs }}` + +| Attribute | Description | +|-----------|-------------| +| `{{ .Title }}` | Document title | +| `{{ .ID }}` | Document identifier | +| `{{ .Content }}` | Document content | +| `{{ .MarkdownContent }}` | Markdown content | +| `{{ .MarkdownEnabled }}` | Boolean for Markdown enablement | + +##### Catalogues + +Accessible via `{{ range .catalogues }}` + +| Attribute | Description | +|-----------|-------------| +| `{{ .Name }}` | Catalogue name | +| `{{ .VisibilityStatus }}` | Catalogue visibility status | + +##### Plans + +Accessible via `{{ range .unique_plans }}` + +| Attribute | Description | +|-----------|-------------| +| `{{ .Name }}` | Plan name | +| `{{ .ID }}` | Plan ID | +| `{{ .DisplayName }}` | Plan display name | +| `{{ .Description }}` | Plan description | +| `{{ .AuthType }}` | Plan authentication type | +| `{{ .Rate }}` | Plan rate | +| `{{ .Per }}` | Plan rate per time unit | +| `{{ .QuotaMax }}` | Plan maximum quota | +| `{{ .QuotaRenewalRate }}` | Plan quota renewal rate | +| `{{ .AutoApproveAccessRequests }}` | Boolean for auto-approval of access requests | + +##### Related Posts + +Accessible via `{{ range .posts }}` + +| Attribute | Description | +|-----------|-------------| +| `{{ .Title }}` | Post title | +| `{{ .Lede }}` | Post summary | +| `{{ .Content }}` | Post content | +| `{{ .MarkdownContent }}` | Markdown content | +| `{{ .MarkdownEnabled }}` | Boolean for Markdown enablement | +| `{{ .Path }}` | Post path | +| `{{ .HeaderImage.URL }}` | Header image URL | +| `{{ .BlogSiteID }}` | Blog site ID | +| `{{ .ProductID }}` | Associated product ID | +| `{{ .AuthorID }}` | Author ID | + +<details> +<summary> <b>Example Usage</b></summary> + +```html +<div class="product-detail"> + <h1>{{ .product.DisplayName }}</h1> + <img src="{{ .product.Logo.URL }}" alt="{{ .product.Name }} logo"> + <p>{{ .product.Description }}</p> + <h2>API Details</h2> + {{ range .product.APIDetails }} + <h3>{{ .Name }}</h3> + <p>Status: {{ .Status }}</p> + <p>Target URL: {{ .TargetURL }}</p> + {{ end }} + <h2>Documentation</h2> + {{ range .product.Docs }} + <h3>{{ .Title }}</h3> + {{ if .MarkdownEnabled }} + {{ .MarkdownContent | markdownify }} + {{ else }} + {{ .Content }} + {{ end }} + {{ end }} + <h2>Available in Catalogues</h2> + <ul> + {{ range .catalogues }} + <li>{{ .Name }} ({{ .VisibilityStatus }})</li> + {{ end }} + </ul> + <h2>Available Plans</h2> + {{ range .unique_plans }} + <div class="plan"> + <h3>{{ .DisplayName }}</h3> + <p>{{ .Description }}</p> + <p>Rate: {{ .Rate }} per {{ .Per }}</p> + <p>Quota: {{ .QuotaMax }}</p> + </div> + {{ end }} + <h2>Related Posts</h2> + {{ range .posts }} + <div class="related-post"> + <h3><a href="{{ .Path }}">{{ .Title }}</a></h3> + <img src="{{ .HeaderImage.URL }}" alt="{{ .Title }}"> + <p>{{ .Lede }}</p> + </div> + {{ end }} +</div> +``` + +</details> + +#### Product OAS Documentation + +**Template Paths**: +- `themes/default/views/product_doc_stoplight_spec.tmpl` +- `themes/default/views/product_doc_redoc.tmpl` + +These templates are used to render the OpenAPI Specification (OAS) documentation for a product. The Stoplight Spec and +ReDoc versions are available. + +##### Available Attributes + +| Attribute | Description | +|-----------|-------------| +| `{{ .Name }}` | Product name | +| `{{ .Description }}` | Product description | +| `{{ .Path }}` | Product path | +| `{{ .Url }}` | OAS document URL | + +<details> +<summary> <b>Example Usage</b></summary> + +```html +<div class="docs-container"> + <div class="card mt-4"> + <div class="card-body"> + <h3 class="card-title"> + <a href="/portal/catalogue-products/{{ .Path }}">{{ .Name }}</a> + </h3> + <p class="card-text"> + {{ .Description }} + </p> + </div> + </div> + <div> + <elements-api + apiDescriptionUrl='{{ .Url }}' + router="hash" + layout="responsive" + /> + </div> +</div> +``` + +</details> + +### Global Helper Functions + +This section provides a detailed overview of the global helper functions available in the Tyk Enterprise Developer +Portal templates. These functions are accessible across the public and private templates and allow you to perform +various operations, retrieve specific data, and create dynamic content within your templates. + +#### Available Functions + +- [CanCreateOrganisation](#cancreateorganisation) +- [Clients](#clients) +- [Current User](#currentuser) +- [FeaturedProducts](#featuredproducts) +- [FilterUserInvites](#filteruserinvites) +- [FormatTime](#formattime) +- [GetCart](#getcart) +- [GetCatalogueList](#getcataloguelist) +- [GetCataloguesForProduct](#getcataloguesforproduct) +- [GetClientDescription](#getclientdescription) +- [GetClientName](#getclientname) +- [GetMenus](#getmenus) +- [GetProducts](#getproducts) +- [IsPortalDisabled](#isportaldisabled) +- [IsPortalPrivate](#isportalprivate) +- [ProductDocRenderer](#productdocrenderer) +- [ProviderUpstreamURL](#providerupstreamurl) +- [SplitStrings](#splitstrings) +- [TruncateString](#truncatestring) +- [TypeOfCredential](#typeofcredential) + +##### CanCreateOrganisation + +Returns true if user can create an organization. + +###### Example Usage +``` +{{ if CanCreateOrganisation req }} + ... +{{ end }} +``` + +##### Clients + +Returns the list of applications for the current user. Expects the request as argument. + +###### Client Attributes + +Accessible via `{{ range $client := Clients req }}` + +| Attribute | Description | +|-----------|-------------| +| `{{ $client.ID }}` | Client ID | +| `{{ $client.Name }}` | Client name | +| `{{ $client.Description }}` | Client description | +| `{{ $client.RedirectURLs }}` | Client redirect URLs | +| `{{ $client.Credentials }}` | Array of client credentials | +| `{{ $client.AccessRequests }}` | Array of client access requests | + +###### Credential Attributes (Within client) + +Accessible via `{{ range $cred := $client.Credentials }}` + +| Attribute | Description | +|-----------|-------------| +| `{{ $cred.ID }}` | Credential ID | +| `{{ $cred.Credential }}` | Credential | +| `{{ $cred.CredentialHash }}` | Credential hash | +| `{{ $cred.OAuthClientID }}` | OAuth client ID | +| `{{ $cred.OAuthClientSecret }}` | OAuth client secret | +| `{{ $cred.Expires }}` | Credential expiration | +| `{{ $cred.AccessRequestID }}` | Access request ID associated with the credential | + +###### Access Request Attributes (Within client) + +Accessible via `{{ range $acreq := $client.AccessRequests }}` + +| Attribute | Description | +|-----------|-------------| +| `{{ $acreq.ID }}` | Access request ID | +| `{{ $acreq.Status }}` | Access request status | +| `{{ $acreq.UserID }}` | User ID associated with access request | +| `{{ $acreq.AuthType }}` | Access request auth type | +| `{{ $acreq.DCREnabled }}` | true if access request DCR enabled | +| `{{ $acreq.ProvisionImmediately }}` | true if provisioned immediately is enabled | +| `{{ $acreq.CatalogueID }}` | Catalogue ID | + +###### Product Attributes (Within access request) + +Accessible via `{{ range $product := $acreq.Products }}` + +| Attribute | Description | +|-----------|-------------| +| `{{ $product.ID }}` | Product ID | +| `{{ $product.Name }}` | Product name | +| `{{ $product.DisplayName }}` | Product display name | +| `{{ $product.Path }}` | Product path | +| `{{ $product.Description }}` | Product description | +| `{{ $product.Content }}` | Product content | +| `{{ $product.AuthType }}` | Product auth type | +| `{{ $product.DCREnabled }}` | true if product DCR enabled | + +###### Plan Attributes (Within access request) + +Accessible via `{{ $acreq.Plan }}` + +| Attribute | Description | +|-----------|-------------| +| `{{ .Name }}` | Plan name | +| `{{ .DisplayName }}` | Plan display name | +| `{{ .Description }}` | Plan description | +| `{{ .AuthType }}` | Plan auth type | +| `{{ .Rate }}` | Plan rate | +| `{{ .Per }}` | Plan period | +| `{{ .QuotaMax }}` | Plan quota maximum | +| `{{ .QuotaRenewalRate }}` | Plan quota renewal rate | +| `{{ .AutoApproveAccessRequests }}` | true if auto-approve access requests is enabled | + +###### Example Usage +```html +{{ range $client := Clients req }} +<div class="client"> + <h2>Client: {{ $client.Name }}</h2> + {{ range $acreq := $client.AccessRequests }} + <h4>Products:</h4> + <ul> + {{ range $product := $acreq.Products }} + <li> + <strong>{{ $product.Name }}</strong> + {{ $product.Description }} + </li> + {{ end }} + </ul> + <h4>Plan:</h4> + <p><strong>Name:</strong> {{ $acreq.Plan.Name }}</p> + <p><strong>Rate:</strong> {{ $acreq.Plan.Rate }} per {{ $acreq.Plan.Per }}</p> + <p><strong>Quota Max:</strong> {{ $acreq.Plan.QuotaMax }}</p> + {{ end }} +</div> +{{ end }} +``` + +##### CurrentUser + +The `CurrentUser` function returns the current user object if a user is logged in. It expects the request as an argument. + +##### User Attributes + +Accessible via `{{ $user := CurrentUser req }}` + +| Attribute/Method | Description | +|-------------------|-------------| +| `{{ $user.ID }}` | User ID | +| `{{ $user.First }}` | User name | +| `{{ $user.Last }}` | User surname | +| `{{ $user.Email }}` | User email | +| `{{ $user.OrganisationID }}` | User organization ID | +| `{{ $user.DisplayName }}` | User complete name | +| `{{ $user.IdentityProvider }}` | User provider (Portal or Tyk Identity Broker) | +| `{{ $user.GetOrganisationID }}` | User's organization ID | +| `{{ $user.IsAdmin }}` | true if user is an admin | +| `{{ $user.IsOrgAdmin }}` | true if user is an organization admin | +| `{{ $user.DisplayRole }}` | User's role | + +###### Example Usage +```html +{{ $user := CurrentUser req }} +{{ if $user }} +<div class="user-info"> + <h2>Welcome, {{ $user.DisplayName }}!</h2> + <p>Email: {{ $user.Email }}</p> + {{ if $user.IsAdmin }} + <p>You have admin privileges.</p> + {{ else if $user.IsOrgAdmin }} + <p>You are an organization admin.</p> + {{ else }} + <p>Your role: {{ $user.DisplayRole }}</p> + {{ end }} +</div> +{{ else }} +<p>Please log in to view your account information.</p> +{{ end }} +``` + +##### FeaturedProducts + +Returns a list of featured products. + +###### Product Attributes + +Accessible via `{{ range FeaturedProducts }}` + +| Attribute | Description | +|-----------|-------------| +| `{{ .ID }}` | Product ID | +| `{{ .Name }}` | Product name | +| `{{ .DisplayName }}` | Product display name | +| `{{ .Path }}` | Product path | +| `{{ .ReferenceID }}` | Product reference ID | +| `{{ .Description }}` | Product description | +| `{{ .AuthType }}` | Product auth type | +| `{{ .Scopes }}` | Product scopes | +| `{{ .Logo.URL }}` | Product logo URL | +| `{{ .Feature }}` | true if the product is featured | +| `{{ .DCREnabled }}` | true if DCR is enabled | +| `{{ .ProviderID }}` | Provider ID | +| `{{ .APIDetails }}` | Array of API details associated with the product | +| `{{ .Catalogues }}` | Array of catalogues associated with the product | + +###### API Details Attributes (Within product) + +Accessible via `{{ range .APIDetails }}` + +| Attribute | Description | +|-----------|-------------| +| `{{ .Name }}` | API name | +| `{{ .Description }}` | API description | +| `{{ .APIType }}` | API type | +| `{{ .TargetURL }}` | API target URL | +| `{{ .ListenPath }}` | API listen path | +| `{{ .OASUrl }}` | API OAS URL | +| `{{ .Status }}` | "Active" if API status is active, otherwise "Inactive" | + +###### Catalogue Attributes (Within product) + +Accessible via `{{ range .Catalogues }}` + +| Attribute | Description | +|-----------|-------------| +| `{{ .Name }}` | Catalogue name | +| `{{ .VisibilityStatus }}` | Catalogue visibility status | + +<details> +<summary> <b>Example Usage</b></summary> + +```html +{{ $featured_products := FeaturedProducts }} +<h2>Featured API Products</h2> +<p>Explore our highlighted API offerings</p> +<div class="featured-products-container"> + {{ range $featured_products }} + <div class="product-card"> + {{ if .Logo }} + <img src="{{ .Logo.URL }}" alt="{{ .Name }} logo"> + {{ end }} + <div class="product-info"> + <span class="auth-type">{{ .AuthType }}</span> + <h3>{{ .Name }}</h3> + <p>{{ .Description }}</p> + </div> + <div class="product-actions"> + <a href="/portal/catalogue-products/{{ .Path }}" class="btn">More Info</a> + <div class="dropdown-content"> + {{ range .APIDetails }} + {{ if or (gt (.OASDocument.Base.Url | trim | length) 0) (gt (.OASUrl | trim | length) 0) }} + <a href="/portal/catalogue-products/{{ $.Path }}/{{ .APIID }}/docs" target="blank"> + {{ .Name }} + </a> + {{ end }} + {{ end }} + </div> + </div> + </div> + {{ end }} +</div> +``` + +</details> + +##### FilterUserInvites + +Returns a list of users that were invited to the current user's organization, if the user became an organization. +Expects the request as a parameter. + +###### User Attributes + +Accessible via `{{ range $invite := FilterUserInvites req }}` + +| Attribute | Description | +|-----------|-------------| +| `{{ $invite.ID }}` | User ID | +| `{{ $invite.Email }}` | User email | +| `{{ $invite.First }}` | User first name | +| `{{ $invite.Last }}` | User last name | +| `{{ $invite.Role }}` | User role | +| `{{ $invite.JoinedAt }}` | User joined at time | +| `{{ $invite.Joined }}` | Whether the user has joined | +| `{{ $invite.Uactive }}` | Whether the user is active | + +<details> +<summary> <b>Example Usage</b></summary> + +```html +{{ $userInvites := FilterUserInvites req }} +{{ if $userInvites }} +<h2>Invited Users</h2> +<table> + <thead> + <tr> + <th>Name</th> + <th>Email</th> + <th>Role</th> + <th>Status</th> + </tr> + </thead> + <tbody> + {{ range $invite := $userInvites }} + <tr> + <td>{{ $invite.First }} {{ $invite.Last }}</td> + <td>{{ $invite.Email }}</td> + <td>{{ $invite.Role }}</td> + <td> + {{ if $invite.Joined }} + Joined + {{ else if $invite.Uactive }} + Pending + {{ else }} + Inactive + {{ end }} + </td> + </tr> + {{ end }} + </tbody> +</table> +{{ else }} +<p>No pending invitations.</p> +{{ end }} +``` + +</details> + +##### FormatTime + +Formats a given time with a given format. + +###### Example Usage +```gotemplate +{{ $user := CurrentUser req }} +{{ if $user}} +{{$time := FormatTime $user.CreatedAt "2 Jan, 2006 at 3:04:00 PM (MST)"}} +<!-- Use $time or other variables here --> +... +{{end}} +``` + +##### GetCart + +Returns a map with the cart items for a given user ID. Expects the user ID as an argument. This function is useful for +retrieving and displaying the contents of a user's cart, including detailed information about the products, their +authentication types, and associated templates. + +###### Cart Item Attributes + +Accessible via `{{ range $key, $value := GetCart $user.ID }}` + +| Attribute | Description | +|-----------|-------------| +| `{{ $value.AuthType }}` | Cart item auth type | +| `{{ $value.Catalogue }}` | Cart item catalogue | +| `{{ $value.DCREnabled }}` | true if cart order consists of DCR products | +| `{{ $value.Plan }}` | Cart item plan | +| `{{ $value.Products }}` | Cart item array of products | + +##### Plan Attributes (Within cart item) + +Accessible via `{{ $plan := $value.Plan }}` + +| Attribute | Description | +|-----------|-------------| +| `{{ .ID }}` | Plan ID | +| `{{ .PlanName }}` | Plan name | +| `{{ .FormatQuota }}` | Formatted quota information | +| `{{ .FormatRateLimit }}` | Formatted rate limit information | + +##### Catalogue Attributes (Within cart item) + +Accessible via `{{ $catalogue := $value.Catalogue }}` + +| Attribute | Description | +|-----------|-------------| +| `{{ .ID }}` | Catalogue ID | + +###### Product Attributes (Within cart item) + +Accessible via `{{ range $product := $value.Products }}` + +| Attribute | Description | +|-----------|-------------| +| `{{ .ID }}` | Product ID | +| `{{ .Name }}` | Product name | +| `{{ .DisplayName }}` | Product display name | +| `{{ .Path }}` | Product path | +| `{{ .Description }}` | Product description | +| `{{ .Content }}` | Product content | +| `{{ .AuthType }}` | Product auth type | +| `{{ .DCREnabled }}` | true if product DCR enabled | +| `{{ .AuthTypes }}` | List of product auth types | + +###### DCR Client Template Attributes (Within product) + +Accessible via `{{ range $template := $product.Templates }}` + +| Attribute | Description | +|-----------|-------------| +| `{{ .ID }}` | Template ID | +| `{{ .Name }}` | Template name | +| `{{ .GrantType }}` | Template grant type | +| `{{ .ResponseTypes }}` | Template response types | +| `{{ .TokenEndpointAuthMethod }}` | Template token endpoint auth method | +| `{{ .OktaAppType }}` | Template Okta app type | + +###### Example Usage +```html +{{ $user := CurrentUser req }} +{{ if $user }} + {{ $cart := GetCart $user.ID }} + {{ if $cart }} + <h2>Your Cart</h2> + {{ range $key, $value := $cart }} + <div class="cart-item"> + <h3>{{ $value.Catalogue.Name }}</h3> + <p>Auth Type: {{ $value.AuthType }}</p> + {{ range $product := $value.Products }} + <div class="product"> + <h4>{{ $product.DisplayName }}</h4> + <p>{{ $product.Description }}</p> + </div> + {{ end }} + </div> + {{ end }} + {{ else }} + <p>Your cart is empty.</p> + {{ end }} +{{ end }} +``` + +##### GetCatalogueList + +Returns a list of catalogue names. Expects the request as parameter. + +###### Example Usage +```gotemplate +{{ range $key, $value := GetCatalogueList req }} +<option value="{{ $key }}" {{ if eq $value.Selected true }} selected {{ end }}>{{ $value.Name }}</option> +{{ end }} +``` + +##### GetCataloguesForProduct + +Returns a list of products for a given user and product ID. Expects the request, a user and a product ID as parameters. + +###### Catalogue Attributes + +Accessible via `{{ range GetCataloguesForProduct req $user $product.ID }}` + +| Attribute | Description | +|-----------|-------------| +| `{{ .VisibilityStatus }}` | Catalogue visibility status | +| `{{ .Name }}` | Catalogue name | + +###### Example Usage +```html +{{ $thisProduct := .product }} +{{ $user := CurrentUser req }} +{{ $catalogues_for_product := GetCataloguesForProduct req $user $thisProduct.ID }} +<h3>Catalogues for {{ $thisProduct.Name }}</h3> +<ul> + {{ range $catalogues_for_product }} + <li> + <strong>{{ .Name }}</strong> + (Visibility: {{ .VisibilityStatus }}) + </li> + {{ end }} +</ul> +``` + +##### GetClientDescription + +Returns an application description given a credential ID. + +###### Example Usage +``` +{{ range $app.Credentials }} +... +{{ GetClientDescription .ID}} +{{end}} +``` + +##### GetClientName + +Returns an application name given a credential ID. + +###### Example Usage +``` +{{ range $app.Credentials }} +... +{{ GetClientName .ID}} +{{end}} +``` + +##### GetMenus + +Returns a map of all [menus]({{< ref "portal/customization#configure-menus" >}}). + +###### Example Usage +```html +{{ if GetMenus.Primary }} + {{ range GetMenus.Primary.Children }} + {{ range .Children }} + <li class="nav-item"> + <a class="dropdown-item" href="{{ .Path }}">{{ .Tag }}</a> + </li> + {{ end }} + {{ end }} +{{ end }} +``` + +##### GetProducts + +Returns the list of products for the current user. Expects the request as an argument. + +###### Product Attributes + +Accessible via `{{ range $product := GetProducts req }}` + +| Attribute | Description | +|-----------|-------------| +| `{{ $product.ID }}` | Product ID | +| `{{ $product.Name }}` | Product name | +| `{{ $product.DisplayName }}` | Product display name | +| `{{ $product.Path }}` | Product path | +| `{{ $product.ReferenceID }}` | Product reference ID | +| `{{ $product.Description }}` | Product description | +| `{{ $product.AuthType }}` | Product auth type | +| `{{ $product.Scopes }}` | Product scopes | +| `{{ $product.Logo.URL }}` | Product logo URL | +| `{{ $product.Feature }}` | true if the product is featured | +| `{{ $product.DCREnabled }}` | true if DCR is enabled | +| `{{ $product.ProviderID }}` | Provider ID | +| `{{ $product.APIDetails }}` | Array of API details associated with the product | +| `{{ $product.Catalogues }}` | Array of catalogues associated with the product | + +###### API Details Attributes (Within product) + +Accessible via `{{ range $api := $product.APIDetails }}` + +| Attribute | Description | +|-----------|-------------| +| `{{ $api.Name }}` | API name | +| `{{ $api.Description }}` | API description | +| `{{ $api.APIType }}` | API type | +| `{{ $api.TargetURL }}` | API target URL | +| `{{ $api.ListenPath }}` | API listen path | +| `{{ $api.OASUrl }}` | API OAS URL | +| `{{ $api.Status }}` | "Active" if API status is active, otherwise "Inactive" | + +###### Catalogue Attributes (Within product) + +Accessible via `{{ range $catalogue := $product.Catalogues }}` + +| Attribute | Description | +|-----------|-------------| +| `{{ $catalogue.Name }}` | Catalogue name | +| `{{ $catalogue.VisibilityStatus }}` | Catalogue visibility status | + +<details> +<summary> <b>Example Usage</b></summary> + +```html +{{ range GetProducts req }} +<div class="col-lg-12 card-container"> + <div class="card d-flex flex-row {{ if .Logo.URL }}has-logo{{ end }}"> + {{ if .Logo.URL }} + <img class="card-img-top img-fluid" src="{{ .Logo.URL }}" alt=""> + {{ end }} + <div class="card-body align-self-center w-100"> + <div class="card-title d-flex flex-column justify-content-end align-items-baseline"> + <div class="pill-container"> + <span class="pill">{{ .AuthType }}</span> + </div> + <h2>{{ .ProductName }}</h2> + </div> + {{ if .Description }} + <p class="card-text">{{ .Description }}</p> + {{ end }} + </div> + <div class="card-cta d-flex flex-column align-self-center justify-content-between align-items-baseline w-100"> + <div> + <a href="/portal/catalogue-products/{{ .Path }}" class="btn btn-secondary">More Info</a> + </div> + </div> + </div> +</div> +{{ end }} +``` + +</details> + +##### IsPortalDisabled + +Returns true (exception: for admins is always enabled) if portal visibility was set to hidden. Expects the request as +parameter. + +###### Example Usage +``` +{{ $portalDisabled := IsPortalDisabled req }} +``` + +##### IsPortalPrivate + +Returns true (exception: for admins is always enabled) if portal visibility was set to private. Expects the request as +parameter. + +###### Example Usage +``` +{{ $portalPrivate := IsPortalPrivate req }} +``` + +##### ProductDocRenderer + +Returns the configured product OAS renderer (redoc or stoplight). + +###### Example Usage +``` +{{ $oas_template := ProductDocRenderer }} +``` + +##### ProviderUpstreamURL + +Returns the provider upstream URL for a given providerID. Expects the request and a provider ID as parameters. + +###### Example Usage +``` +{{ $upstreamURL := ProviderUpstreamURL req $thisProduct.ProviderID }} +``` + +##### SplitStrings + +Splits a given string with given separator and returns a slice of split strings. + +###### Example Usage +``` +{{ range $app.Credentials }} +... +{{ range SplitStrings .GrantType "," }} +... +{{ end }} +{{ end }} +``` + +##### TruncateString + +Truncates a given string to a given length, returning the truncated string followed by three dots (…). + +###### Example Usage +``` +{{ TruncateString $api.Description 60 }} +``` + +##### TypeOfCredential + +Returns the credential type ("oAuth2.0" or "authToken") given the credential. + +###### Example Usage +``` +{{ range $app.Credentials }} +... +{{ if eq (TypeOfCredential . ) "oAuth2.0" }} +... +{{ end }} +{{end}} +``` + + +### Email Templates + +This section provides a detailed overview of the email template data available in the Tyk Enterprise Developer Portal. +The Tyk Enterprise Developer Portal uses a variety of email templates for different purposes, such as user registration +and access request status or organization status updates. Each template has access to specific data or functions relevant +to its purpose. + +It's important to note that while email templates can include template data or specific template functions, they do not +have access to the global helper functions available in other portal templates. + +Please refer to [email workflow]({{< ref "portal/customization#configure-email-notifications" >}}) +for additional detail on email notifications sent by the portal. + + +#### Available Email Templates + +- [Access Request Approve/Reject](#access-request-approvereject) +- [Access Request Submitted](#access-request-submitted) +- [Activate and Deactivate](#activate-and-deactivate) +- [New User Request](#new-user-request) +- [Organization Approve](#organization-approve) +- [Organization Reject](#organization-reject) +- [Organization Request](#organization-request) +- [Reset Password](#reset-password) +- [Targeted Invite](#targeted-invite) +- [Welcome User](#welcome-user) + +##### Access Request Approve/Reject + +**Template Paths**: +- `themes/default/mailers/approve.tmpl` +- `themes/default/mailers/reject.tmpl` + +These templates are used for sending notifications to users when their access requests are approved or rejected. + +###### Available Objects + +There's no data sent to these templates. + +###### Example Usage +``` +Hi, +The API Credentials you provisioned have been rejected. +Thanks, +The Team +``` + +##### Access Request Submitted + +**Template Path**: `themes/default/mailers/submitted.tmpl` + +This template is used for notifying administrators about pending access requests. + +###### Available Objects + +- `{{ .requests }}`: Returns the list of access requests pending approval. + +###### Access Request Attributes + +Accessible via `{{ range .requests }}` + +| Attribute | Description | +|-----------|-------------| +| `{{ .PlanID }}` | Plan ID associated with access request | +| `{{ .Status }}` | Request status | +| `{{ .AuthType }}` | Request authentication type | +| `{{ .UserID }}` | User ID associated with the request | +| `{{ .ClientID }}` | Client ID associated with the request | +| `{{ .DCREnabled }}` | Indicates if DCR (Dynamic Client Registration) is enabled for the request | +| `{{ .ProvisionImmediately }}` | Indicates if provisioning is immediate for the request | +| `{{ .CatalogueID }}` | Catalogue ID associated with the request | + +###### Product Attributes (within Access Request) + +Accessible via `{{ range $product := $acreq.Products }}` + +| Attribute | Description | +|-----------|-------------| +| `{{ $product.ID }}` | Product ID | +| `{{ $product.Name }}` | Product name | +| `{{ $product.DisplayName }}` | Product display name | +| `{{ $product.Description }}` | Product description | +| `{{ $product.AuthType }}` | Product authentication type | +| `{{ $product.DCREnabled }}` | Indicates if DCR (Dynamic Client Registration) is enabled for the product | + +###### Example Usage +```html +<p>A new Access request has been submitted. Please log in to the administration dashboard to view the request.</p> +<ul> + {{ range $acreq := .requests }} + <li> + <strong>Status:</strong> {{ $acreq.Status }}<br> + <strong>User ID:</strong> {{ $acreq.UserID }}<br> + <strong>Products:</strong> + <ul> + {{ range $product := $acreq.Products }} + <li>{{ $product.DisplayName }} ({{ $product.AuthType }})</li> + {{ end }} + </ul> + </li> + {{ end }} +</ul> +``` + + +##### Activate and Deactivate + +**Template Paths**: +- `themes/default/mailers/activate.tmpl` +- `themes/default/mailers/deactivate.tmpl` + +These templates are used for sending activation and deactivation notifications to users. + +###### Available Objects + +- `{{ .name }}`: Returns the user's full name. + +###### Example Usage +``` +Hi, <strong>{{.name}}</strong><br/> +Your account has been activated. +``` + +##### New User Request + +**Template Path**: `themes/default/mailers/newuser.tmpl` + +This template is used for notifying administrators about new user registration requests pending activation. + +###### Available Objects + +- `{{ .user }}`: Returns the new user pending activation. + +##### User Attributes + +Accessible via `{{ .user }}` + +| Attribute/Method | Description | +|-------------------|-------------| +| `{{ .ID }}` | User ID | +| `{{ .First }}` | User name | +| `{{ .Last }}` | User surname | +| `{{ .Email }}` | User email | +| `{{ .OrganisationID }}` | User organization ID | +| `{{ .DisplayName }}` | User complete name | +| `{{ .IdentityProvider }}` | User provider (Portal or Tyk Identity Broker) | +| `{{ .GetOrganisationID }}` | User's organization ID | +| `{{ .IsAdmin }}` | true if user is an admin | +| `{{ .IsOrgAdmin }}` | true if user is an organization admin | +| `{{ .DisplayRole }}` | User's role | +| `{{ .Organisation.Name }}` | Organization name | +| `{{ .Teams }}` | Array of user teams | +| `{{ .Teams.ID }}` | Team ID | +| `{{ .Teams.Name }}` | Team name | +| `{{ .Teams.Default }}` | Indicates if the team is the default team (true/false) | + +###### Example Usage +``` +<p>There is a new user request pending. Please approve it from the admin console.</p> +<p> + Id: {{ .user.ID }}<br/> + User: {{ .user.DisplayName }} ({{ .user.Email }})<br/> + Role: {{ .user.Role }}<br/> + {{ if gt .user.OrganisationID 0 }} + Organisation: {{ .user.Organisation.Name }}<br/> + {{ else }} + Organisation: Administrators' organisation<br/> + {{ end }} + {{ if gt (len .user.Teams) 0 }} + Teams:<br/> + <ul> + {{ range .user.Teams }} + <li>{{ .Name }}</li> + {{ end }} + </ul> + {{ else }} + Teams: none + {{ end }} +</p> +``` + +##### Organization Approve + +**Template Path**: `themes/default/mailers/organisation_request.tmpl` + +This template is used for notifying users that their organization creation request has been approved. + +###### Available Objects + +- `{{ site }}`: Returns the application host. + +###### Example Usage +``` +Hello, +The organization registration request has been approved. You can now manage your organization in your dashboard here: https://{{.site}}/portal/private/dashboard +Thanks, +The team +``` + +##### Organization Reject + +**Template Path**: `themes/default/mailers/organisation_reject.tmpl` + +This template is used for notifying users that their organization creation request has been rejected. + +###### Available Objects + +There's no data sent to this template. + +###### Example Usage +``` +Hello, +The organization registration request has been rejected. +Thanks, +The team +``` + +##### Organization Request + +**Template Path**: `themes/default/mailers/organisation_request.tmpl` + +This template is used for notifying administrators about new organization creation requests. + +###### Available Objects + +- `{{ .user }}`: Returns the user who made the request. +- `{{ .organisationName }}`: Returns the new organization name. + +##### User Attributes + +Accessible via `{{ .user }}` + +| Attribute/Method | Description | +|-------------------|-------------| +| `{{ .ID }}` | User ID | +| `{{ .First }}` | User name | +| `{{ .Last }}` | User surname | +| `{{ .Email }}` | User email | +| `{{ .OrganisationID }}` | User organization ID | +| `{{ .DisplayName }}` | User complete name | +| `{{ .IdentityProvider }}` | User provider (Portal or Tyk Identity Broker) | +| `{{ .GetOrganisationID }}` | User's organization ID | +| `{{ .IsAdmin }}` | true if user is an admin | +| `{{ .IsOrgAdmin }}` | true if user is an organization admin | +| `{{ .DisplayRole }}` | User's role | + +###### Example Usage +``` +There is a new organization registration request pending. Please approve it from the admin console. +The organization name: {{ .organisationName }}. +The user: {{ .user.DisplayName }} ({{ .user.Email }}). +``` + +##### Reset Password + +**Template Path**: `themes/default/mailers/auth/reset_password.tmpl` + +This template is used for sending password reset emails to users. + +###### Available Functions + +- `{{ current_user }}`: Returns the current user object. +- `{{ reset_password_url }}`: Returns the URL with the token for setting the password. + +###### User Attributes + +Accessible via `{{ current_user }}` + +| Attribute/Method | Description | +|-------------------|-------------| +| `{{ .ID }}` | User ID | +| `{{ .First }}` | User name | +| `{{ .Last }}` | User surname | +| `{{ .Email }}` | User email | +| `{{ .Role }}` | User role | +| `{{ .OrganisationID }}` | User organization ID | +| `{{ .DisplayName }}` | User complete name | +| `{{ .IdentityProvider }}` | User provider (Portal or Tyk Identity Broker) | +| `{{ .GetOrganisationID }}` | User's organization ID | +| `{{ .IsAdmin }}` | true if user is an admin | +| `{{ .IsOrgAdmin }}` | true if user is an organization admin | +| `{{ .DisplayRole }}` | User's role | + +###### Example Usage +``` +{{ $user := current_user}} +<p>Hello {{ $user.DisplayName }},</p> +<p>Someone has requested a link to change your password. You can do this through the link below.</p> +<p>{{reset_password_url}}</p> +<p>If you didn't request this, please ignore this email.</p> +<p>Your password won't change until you access the link above and create a new one.</p> +``` + +##### Targeted Invite + +**Template Path**: `themes/default/mailers/auth/targeted_invite.tmpl` + +This template is used for sending targeted invitations to users. + +###### Available Functions + +- `{{ user }}`: Returns the targeted user object. +- `{{ team }}`: Returns the team name to which the user is being invited. +- `{{ invite_url }}`: Returns the URL with the token for setting the password. + +###### User Attributes + +Accessible via `{{ user }}` + +| Attribute/Method | Description | +|-------------------|-------------| +| `{{ .ID }}` | User ID | +| `{{ .First }}` | User name | +| `{{ .Last }}` | User surname | +| `{{ .Email }}` | User email | +| `{{ .Role }}` | User role | +| `{{ .OrganisationID }}` | User organization ID | +| `{{ .DisplayName }}` | User complete name | +| `{{ .IdentityProvider }}` | User provider (Portal or Tyk Identity Broker) | +| `{{ .GetOrganisationID }}` | User's organization ID | +| `{{ .IsAdmin }}` | true if user is an admin | +| `{{ .IsOrgAdmin }}` | true if user is an organization admin | +| `{{ .DisplayRole }}` | User's role | + +###### Example Usage +```html +{{ $u := user }} +Hi, <strong>{{ $u.DisplayName }}</strong><br/> +<p>Someone is inviting you to join {{ if $u.IsAdmin }}as an Administrator{{ else }}the {{ team }} team{{end }}. You can do this through the link below.</p> +<p>{{ invite_url }}</p> +<p>If you didn't request this, please ignore this email.</p> +``` + +##### Welcome User + +**Template Paths**: +- `themes/default/mailers/welcome_admin.tmpl` +- `themes/default/mailers/welcome_dev.tmpl` + +These templates are used for sending welcome emails to new users, with separate templates for administrators and developers. + +###### Available Objects + +- `{{ .user }}`: Returns the user who made the request. Refer to the CurrentUser section for accessible attributes and methods. + +##### User Attributes + +Accessible via `{{ .user }}` + +| Attribute/Method | Description | +|-------------------|-------------| +| `{{ .ID }}` | User ID | +| `{{ .First }}` | User name | +| `{{ .Last }}` | User surname | +| `{{ .Email }}` | User email | +| `{{ .OrganisationID }}` | User organization ID | +| `{{ .DisplayName }}` | User complete name | +| `{{ .IdentityProvider }}` | User provider (Portal or Tyk Identity Broker) | +| `{{ .GetOrganisationID }}` | User's organization ID | +| `{{ .IsAdmin }}` | true if user is an admin | +| `{{ .IsOrgAdmin }}` | true if user is an organization admin | +| `{{ .DisplayRole }}` | User's role | +| `{{ .Organisation.Name }}` | organization name | +| `{{ .Teams }}` | Array of user teams | +| `{{ .Teams.ID }}` | Team ID | +| `{{ .Teams.Name }}` | Team name | +| `{{ .Teams.Default }}` | Indicates if the team is the default team (true/false) | + +<details> +<summary> <b>Example Usage</b></summary> + +```html +<h1>Welcome to Tyk Enterprise Developer Portal</h1> +<p>Hello {{ .user.DisplayName }},</p> +<p>Your account has been created for the {{ .user.Organisation.Name }} organisation.</p> +<p>Your assigned teams:</p> +<ul> + {{ range .user.Teams }} + <li>{{ .Name }}{{ if .Default }} (Default){{ end }}</li> + {{ end }} +</ul> +<p>We're excited to have you on board!</p> +``` + +</details> + +## Configure Pages + +We suggest you read the [portal concepts]({{< ref "portal/overview#developer-portal-concepts" >}}) first. If you've already done that, we will show you an example on how to: + +- create a new layout. +- create a new template from the file system. +- create a new page from the admin dashboard. + +**Prerequisites** + +- Access to the file system +- Access to your Tyk Self-Managed installation + +### Create a New Page using an Existing Template + +Follow the example below to create a new page called “My first page” using an existing template. + +{{< img src="/img/dashboard/portal-management/enterprise-portal/create-new-page1.png" alt="The pages section within the Tyk Enterprise Portal app" >}} + +1. Log in to the Admin Dashboard. +2. Navigate to Pages from the side bar menu. +3. Click Add and enter the following values: + +{{< img src="/img/dashboard/portal-management/enterprise-portal/add-a-content-page-using-an-existing-template.png" alt="Add a new content page" >}} + +### Create a New Page + +#### Create the Layout File + +A layout behaves like a component that can be reused to inject templates in order to avoid duplicating elements such as `<head>` and `<link>`.So let’s create one that looks like the one below. + +1. From the file system navigate to `/themes/default/layouts`. +2. Create a new file named `my_layout.tmpl`. +3. Copy the code below, paste it to your new layout file and save it. + +```html +<!DOCTYPE html> +<html lang="en"> + + <head> + + <meta charset="utf-8"> + <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no"> + <meta name="description" content=""> + <meta name="author" content=""> + + <title>{{ if .page}} {{.page.Title}} {{else}} Developer Portal {{end}} + + + + + + + + + + {{ render "top_nav" . }} +
+ + {{ yield }} +
+ + + {{ render "footer" . }} + + + + +``` +{{< note success >}} +**Note** + +The above snippet is taking into account the default developer portal setup, directories and references of the appropriate styling files. + +{{< /note >}} + +We have also added the top and footer navigation menus for demonstration purposes, `{{ render "top_nav" . }}` and `{{ render "footer" . }}` respectively. + +#### Create the Template File + +{{< note success >}} +**Note** + +Only follow this step after creating a new template file in Section 1 above, unless you want to use an existing template. + +{{< /note >}} + +1. From the file system; navigate to `/themes/default/views`. +2. Create a new file (template) +3. In this example, we will keep it simple and create a couple of HTML tags. Copy and paste the following code: + +```go +
+

{{ .page.Title }}

+

{{ .blocks.Description.Content }}

+
+``` +In this example, we use the code references in the template: +- Inside the template’s `

` we use `{{ .page.Title }}` to display the page name. +- Inside the template’s `

` we use `{{ .blocks.Description.Content }}` to display the page content. + +4. Name this `my_template.tmpl` and save it. +5. You now need to reference your new layout and template. From your Manifest file (`themes.json`) add a new key to the object that will look like this: + +```json +{ + "name": "My new Template", + "template": "my_template", + "layout": "my_layout" +} +``` +Alternatively, you can give it your preferred name but reference the template and layout names we created earlier in the manifest file. + +6. Now restart your developer portal service and go through the tutorial on how to add a new page. The template dropdown within the add new page form should have a new entry called `My new Template`, so create a page that will look like this: + +{{< img src="/img/dashboard/portal-management/enterprise-portal/add-page-details.png" alt="Add new page details" >}} + +7. Now navigate to the path we have entered on your browser (`my-domain.io/my-first-page`) and you should be able to see your new page with the content added via the UI: + +{{< img src="/img/dashboard/portal-management/enterprise-portal/my-first-page-example.png" alt="Example new UI page" >}} + +Taking as a scenario our above example, let’s see a visual explaining the correlation between the different files and UI. + +{{< img src="/img/dashboard/portal-management/enterprise-portal/scenario-correlation-diagram.png" alt="Scenario correlation" >}} + + +### Edit Page Content + +Content managers or similar roles that are responsible for the content displayed on the developer portal can edit and manage pages within the **Page section** of the Developer Portal. + +**Prerequisites** + +- Install the Developer portal +- Log in to the portal dashboard +- Default pages available or a page created by a developer which has the custom theme linked to it + +#### Instructions + +1. From the Pages section, open an existing page. This could be a page based on the default template or a new page that one of your developers set up when creating a new custom template. +2. Edit the page meta data. Change the name of the page if needed. Set or change the path URL if needed. + +{{< img src="/img/dashboard/portal-management/enterprise-portal/page-content-edit.png" alt="Edit Portal page content" >}} + +3. Edit the page content within the existing content blocks. + +{{< note success >}} +**Note** + +The content block name is linked to the content block name in the template html file. If this name is changed, and not reflected in the template html, the page will get an error and won’t show. + +{{< /note >}} + +4. Publish the page and view it from the external portal URL. If you want the page to be published and visible on the external portal, you need to change the state to Published. + + + +## Configure Menus + +The Developer portal has two types of menus: +1. The main navigation at the top (in the header) +2. The footer at the bottom. + +Both of them are defined as [partials]({{< ref "portal/customization#file-structure-of-a-theme" >}}) in the portal directory in `/themes/default/partials/`. + +### Top Navigation Menu + +The Enterprise Developer portal enables admin users to customize the navigation menu that appears on the top navigational bar of the live portal. An admin user can create and manage menu items without any code from the admin dashboard of the Developer portal. +{{< img src="img/dashboard/portal-management/enterprise-portal/top-nav-menu.png" alt="The navigation menu" >}} + +Each menu item may: +- lead to a specific page or URL: + {{< img src="img/dashboard/portal-management/enterprise-portal/regular-menu-item.png" alt="Regular menu item" >}} +- show a dropdown list with possible navigational options: + {{< img src="img/dashboard/portal-management/enterprise-portal/dropdown-menu-item.png" alt="Dropdown menu item" >}} + +Admin users can create additional navigational menus and render them on any page of the live portal. This customization requires changes to a theme and is covered in the [Full customization section]({{< ref "portal/customization#configure-menus" >}}). + +#### Manage Menu Items + +The management of the menu items is done from the **Menus** section of the Developer portal. + +1. Open the admin dashboard. Navigate to the **Menus** section. + {{< img src="img/dashboard/portal-management/enterprise-portal/navigation-to-menus-section.png" alt="Navigate to the Menus section" >}} + +2. Select a menu that you want to modify. By default, the Developer portal has only one **Primary** menu. If you want to add more menus and render them on the live portal, please refer to [Full customization section]({{< ref "portal/customization#configure-menus" >}}). + {{< img src="img/dashboard/portal-management/enterprise-portal/select-a-menu.png" alt="Select a menu" >}} + +3. Click on a **menu item** to modify it. You can change the following items: + 1. **Title** that will be exposed to developers. + 2. **Path** where developers will be redirected by clicking on that menu item. + 3. **Children** items that will be exposed in the dropdown list that will appear when hovering mouse over the menu item. + 4. To make the changes effectively, you need to save the changes by clicking on the **Save changes** button. + {{< img src="img/dashboard/portal-management/enterprise-portal/menu-item.png" alt="Modify a menu item" >}} + +4. To remove a menu item from the menu click on the **bin** icon and click on the **Save changes** button. + {{< img src="img/dashboard/portal-management/enterprise-portal/delete-a-menu-item.png" alt="Delete a menu item" >}} + +#### Create New Menu Items +To create a new menu item, you need to: + +1. Click on the **Add Menu Item** button. +2. Fill **Title**, **Path**, and **Children** fields. Save the changes by clicking on the **Save changes** button. + {{< img src="img/dashboard/portal-management/enterprise-portal/save-new-menu-item.png" alt="Save a menu item" >}} + +The new menu item will appear on the live portal immediately. +{{< img src="img/dashboard/portal-management/enterprise-portal/new-menu-item-on-the-live-portal.png" alt="New menu item on the live portal" >}}s + +#### Update Existing Menus + +1. Log into your portal +2. Select **Menus** from the navigation menu +3. Click **Primary** to edit the menu + +{{< img src="/img/dashboard/portal-management/enterprise-portal/edit-menu.png" alt="Edit Menu dialog" >}} + +**Field Descriptions** + +- **Name**: You can give it any name you like, it does not have any effect in the live portal nor the admin app. +- **Path**: This will be used in the code as a reference in order to render the menu. If you don’t have access to the template files, we recommend that you do not edit this field. Editing the `Path` for the default menus will hide the menu as there will be a mismatch between the Path and the reference in the template. +- **Menu Items**: + 1. **Title**: This will be the text that will be displayed in the live portal. + 2. **Path**: this is where the user will be redirected to. + 3. **Children**: In this section you add another nested menu item. We have added a dummy item (Product 1) to demonstrate + +Below is the menu item from its own view, which is available from the **Menu Items** option in the admin app side menu. + +{{< img src="/img/dashboard/portal-management/enterprise-portal/edit-menu-item.png" alt="Edit Menu item dialog" >}} + +Here's the menu as displayed in the app: + +{{< img src="/img/dashboard/portal-management/enterprise-portal/portal-menu-live.png" alt="Live menu in app" >}} + +We have mentioned above the relationship between a menu’s `Path` and the code reference in the menu template. Let’s see how the main menu template looks like (the file is `/themes/default/partials/` directory and is called `top_nav.tmpl`) for the part that we are interested in: + +```go +{{ if GetMenus.Primary }} + {{ range GetMenus.Primary.Children }} +

  • + + {{ if .Children }} +
      + {{ range .Children }} +
    • + {{.Tag}} +
      • + {{ end }} +
    + {{ end }} +
    • + {{ end }} +{{ end }} +``` +Let's pick each line that is used to render the menu attributes and see how they work: + +1. `{{ if GetMenus.Primary }}`: This statement calls the “GetMenus” function and checks if there is a menu called `Primary`. If present, it goes into the next line: +2. `{{ range GetMenus.Primary.Children }}` Each Menu (Primary) has some children (Menu items) so what this code does is loop through all the children and they are rendered as below: + +```go +
  • + +``` +Where: + +- `{{ .Path }}` is the Path we have defined from the UI and +- `{{ .Tag }}` is the Name we have defined from the UI. + +So this will render all the menu items (Catalogs - as per screenshot) of the menu (Primary - the name we’ve given to the menu). + +3. `{{ if .Children }}`: This line checks if the menu item has any submenus. If it does it loops through those children `{{ range .Children }}` and finally renders them `{{.Tag}}` similarly as the main menu items. +So now the child of **Catalogs** which we named **Product 1** has been rendered. + +## Configure Branding + +In this section we will explain how to apply your branding (styling - CSS) on the portal elements with your own colors and logo within minutes. + +**Prerequisites** + +- A Tyk Self-Managed [installation]({{< ref "tyk-self-managed#installation-options-for-tyk-self-managed" >}}) +- A login for the portal admin app +- Access to your Tyk portal file system + +### Changing Portal Logo + +1. Access the file directory for the Developer portal +2. The default logo is located in `/themes/default/assets/images/` and is called `dev-portal-logo.svg`. +3. Replace the default image with your own, keeping the same file name and in `.svg` format, ensuring that `xmlns="http://www.w3.org/2000/svg"` is included within your `` tag. + +{{< note success >}} +**Note** + +If you want to use different naming, path reference or extension, the code is `` and is found on line 6 from the `/themes/default/partials/footer.tmpl` template. +{{< /note >}} + +### Changing Brand Colors + +Let’s now explain how to manage borders and change the colors of buttons, texts and backgrounds. The file we’ll be looking at is `/themes/default/assets/stylesheets/main.css` which contains some CSS variables that are used throughout the app. Let’s take a closer look. +You can apply some changes in the portal based on your preferences. For example, you can change the navigation background color, the text color and the different button theme colors. Furthermore, you can change table border color and radius. + +If you want to change the navigation background color you need to edit the variable called `--tdp-nav-bg-color` Similarly other variables as you can see where/how each one is used: + +{{< note success >}} +**Note** + +`tdp` stands for Tyk Developer Portal + +{{< /note >}} + +#### Background Colors + +{{< img src="/img/dashboard/portal-management/enterprise-portal/background-colors.png" alt="Background Colour settings Tyk Enterprise Portal" >}} + +- `--tdp-nav-bg-color` navigation background color +- `--tdp-body-bg-color` App background color + +#### Text Colors + +{{< img src="/img/dashboard/portal-management/enterprise-portal/text-colors.png" alt="Text Colour settings Tyk Enterprise Portal" >}} + +- `--tdp-text-color` default text color +- `--tdp-link-color` links (anchor tags) +- `--tdp-nav-link-color` navigation links + +#### Borders + +{{< img src="/img/dashboard/portal-management/enterprise-portal/borders.png" alt="Border Colour settings Tyk Enterprise Portal" >}} + +- `--tdp-card-border-radius` Card component +- `--tdp-border-color-on-error` input color if there’s an error +- `--tdp-table-border-color` table +- `--tdp-border-radius` radius +- `--tdp-primary-border form` elements (such as `` and `` and ` - - {{ range $app := .apps }} - - {{ end }} - -``` - -## Application Create - -**Template Path**: `themes/default/views/app_form_create.tmpl` - -This template is used to render the application creation form. - -### Available Objects - -- `{{ .errors }}`: Map of template errors (Key: category, Value: error message) - -#### Example Usage -```html -{{ if .errors }} -{{ range $key, $errs := .errors }} - -{{ end }} -{{ end }} -``` - -## Application Detail - -**Template Path**: `themes/default/views/app_form_update.tmpl` - -This template is used to render the application detail and update form. - -### Available Objects - -- `{{ .errors }}`: Map of template errors (Key: category, Value: error message) -- `{{ .app }}`: Selected application object. -- `{{ .appCerts }}`: Map of application MTLS certificates if applicable (Key: access request ID, Value: certificate) -- `{{ .allCerts }}`: Map of all MTLS certificates stored if applicable (Key: cert fingerprint, Value: cert) - -### MTLS Certificate Attributes (appCerts) - -| Attribute | Description | -|-----------|-------------| -| `{{ .ID }}` | Certificate ID | -| `{{ .Name }}` | Certificate name | -| `{{ .Fingerprint }}` | Certificate fingerprint | -| `{{ .SignatureAlgorithm }}` | Signature algorithm | -| `{{ .Issuer }}` | Certificate issuer | -| `{{ .IsValid }}` | Boolean indicating if the certificate is valid | -| `{{ .ValidNotBefore }}` | Start date of validity | -| `{{ .ValidNotAfter }}` | End date of validity | -| `{{ .Subject }}` | Certificate subject | - -### MTLS Certificate Attributes (allCerts) - -| Attribute | Description | -|-----------|-------------| -| `{{ .ID }}` | Certificate ID | -| `{{ .Name }}` | Certificate name | - -### Client Attributes - -Accessible via `{{ .app }}` - -| Attribute | Description | -|-----------|-------------| -| `{{ .ID }}` | Client ID | -| `{{ .Name }}` | Client name | -| `{{ .Description }}` | Client description | -| `{{ .RedirectURLs }}` | Client redirect URLs | -| `{{ .Credentials }}` | Array of client credentials | -| `{{ .AccessRequests }}` | Array of client access requests | - -### Client Credentials Attributes - -Accessible via `{{ range $cred := .app.Credentials }}` - -| Attribute | Description | -|-----------|-------------| -| `{{ .ID }}` | Credential ID | -| `{{ .Credential }}` | Credential | -| `{{ .CredentialHash }}` | Credential hash | -| `{{ .OAuthClientID }}` | OAuth client ID | -| `{{ .OAuthClientSecret }}` | OAuth client secret | -| `{{ .Expires }}` | Credential expiration | -| `{{ .AccessRequestID }}` | Access request ID associated with the credential | - -### Client Access Requests Attributes - -Accessible via `{{ range $acreq := .app.AccessRequests }}` - -| Attribute | Description | -|-----------|-------------| -| `{{ .ID }}` | Access request ID | -| `{{ .Status }}` | Access request status | -| `{{ .UserID }}` | User ID associated with access request | -| `{{ .AuthType }}` | Access request auth type | -| `{{ .DCREnabled }}` | true if access request DCR enabled | -| `{{ .ProvisionImmediately }}` | true if provisioned immediately is enabled | -| `{{ .CatalogueID }}` | Catalogue ID | - -### Product Attributes (within Access Request) - -Accessible via `{{ $product := $acreq.Product }}` - -| Attribute | Description | -|-----------|-------------| -| `{{ .ID }}` | Product ID | -| `{{ .Name }}` | Product name | -| `{{ .DisplayName }}` | Product display name | -| `{{ .Path }}` | Product path | -| `{{ .Description }}` | Product description | -| `{{ .Content }}` | Product content | -| `{{ .AuthType }}` | Product auth type | -| `{{ .DCREnabled }}` | true if product DCR enabled | - -### Plan Attributes (within Access Request) - -Accessible via `{{ $acreq.Plan }}` - -| Attribute | Description | -|-----------|-------------| -| `{{ .Name }}` | Plan name | -| `{{ .DisplayName }}` | Plan display name | -| `{{ .Description }}` | Plan description | -| `{{ .AuthType }}` | Plan auth type | -| `{{ .Rate }}` | Plan rate | -| `{{ .Per }}` | Plan period | -| `{{ .QuotaMax }}` | Plan quota maximum | -| `{{ .QuotaRenewalRate }}` | Plan quota renewal rate | -| `{{ .AutoApproveAccessRequests }}` | true if auto-approve access requests is enabled | - -#### Example Usage -```html -

    {{ .app.Name >}}

    -

    {{ .app.Description }}

    -

    Credentials

    -{{ range $cred := .app.Credentials }} -
    -

    ID: {{ $cred.ID }}

    -

    OAuth Client ID: {{ $cred.OAuthClientID }}

    -

    Expires: {{ $cred.Expires }}

    -
    -{{ end }} -

    Access Requests

    -{{ range $acreq := .app.AccessRequests }} -
    -

    ID: {{ $acreq.ID }}

    -

    Status: {{ $acreq.Status }}

    -

    Product: {{ $acreq.Product.Name }}

    -

    Plan: {{ $acreq.Plan.Name }}

    -
    -{{ end }} -``` - -## Blogs - -**Template Path**: `themes/default/views/blog_listing.tmpl` - -This template is used to render the blog listing page. - -### Available Objects - -- `{{ .posts }}`: List of all published blog posts - -### Blog Attributes - -Accessible via `{{ range .posts }}` - -| Attribute | Description | -|-----------|-------------| -| `{{ .Title }}` | Blog post title | -| `{{ .Lede }}` | Blog post summary | -| `{{ .Content }}` | Blog post content | -| `{{ .MarkdownContent }}` | Markdown content | -| `{{ .MarkdownEnabled }}` | Boolean for Markdown enablement | -| `{{ .Path }}` | Blog post path | -| `{{ .HeaderImage.URL }}` | Header image URL | -| `{{ .BlogSiteID }}` | Blog site ID | -| `{{ .ProductID }}` | Associated product ID | -| `{{ .AuthorID }}` | Author ID | -| `{{ .URL }}` | Full URL of the blog post | - -#### Example Usage -```html -

    Blog Posts

    -{{ range .posts }} -
    -

    {{ .Title }}

    - {{ .Title }} -

    {{ .Lede }}

    -
    -{{ end }} -``` - -## Blog Detail - -**Template Path**: `themes/default/views/blog_detail.tmpl` - -This template is used to render the blog detail page. - -### Available Objects - -- `{{ .post }}`: The selected blog post object. -- `{{ .latest_posts }}`: List of 3 latest blog posts. - -### Blog Attributes - -Accessible via `{{ .post }}` or `{{ range .latest_posts }}` - -| Attribute | Description | -|-----------|-------------| -| `{{ .Title }}` | Blog post title | -| `{{ .Lede }}` | Blog post summary | -| `{{ .Content }}` | Blog post content | -| `{{ .MarkdownContent }}` | Markdown content | -| `{{ .MarkdownEnabled }}` | Boolean for Markdown enablement | -| `{{ .Path }}` | Blog post path | -| `{{ .HeaderImage.URL }}` | Header image URL | -| `{{ .BlogSiteID }}` | Blog site ID | -| `{{ .ProductID }}` | Associated product ID | -| `{{ .AuthorID }}` | Author ID | -| `{{ .URL }}` | Full URL of the blog post | - -#### Example Usage -``` -

    {{ .post.Title }}

    -{{ .post.Title }} -

    {{ .post.Lede }}

    -{{ if .post.MarkdownEnabled }} -{{ .post.MarkdownContent | markdownify }} -{{ else }} -{{ .post.Content }} -{{ end }} -

    Read more at: {{ .post.URL }}

    -

    Latest Posts

    -{{ range .latest_posts }} -
    -

    {{ .Title }}

    -

    {{ .Lede }}

    -
    -{{ end }} -``` - -## Cart Checkout - -**Template Path**: `themes/default/views/portal_checkout.tmpl` - -This template is used to render the cart checkout page. - -### Available Objects - -- `{{ .cart }}`: Map with the cart items for the current user -- `{{ .apps }}`: List of applications for the current user -- `{{ .catalogue_count }}`: Cart catalogues count -- `{{ .certs }}`: List of MTLS certificates if applicable -- `{{ .errors }}`: Map of template errors (Key: category, Value: error message) -- `{{ .provisioned }}`: Boolean indicating whether an access request has been provisioned for the cart - -### Application Attributes - -Accessible via `{{ range .apps }}` - -| Attribute | Description | -|-----------|-------------| -| `{{ .Name }}` | Application name | -| `{{ .Description }}` | Application description | -| `{{ .RedirectURLs }}` | Application redirect URLs | - -### MTLS Certificate Attributes - -Accessible via `{{ range .certs }}` - -| Attribute | Description | -|-----------|-------------| -| `{{ .ID }}` | Certificate ID | -| `{{ .Name }}` | Certificate name | - -### Cart Item Attributes - -Accessible via `{{ range $key, $value := .cart }}` - -| Attribute | Description | -|-----------|-------------| -| `{{ $value.AuthType }}` | Cart item auth type | -| `{{ $value.Catalogue }}` | Cart item catalogue | -| `{{ $value.Products }}` | Cart item array of products | -| `{{ $value.Plan }}` | Cart item plan | -| `{{ $value.DCREnabled }}` | true if cart order consists of DCR products | - -### Plan Attributes (Within cart item) - -Accessible via `{{ $plan := $value.Plan }}` - -| Attribute | Description | -|-----------|-------------| -| `{{ .ID }}` | Plan ID | -| `{{ .PlanName }}` | Plan name | -| `{{ .FormatQuota }}` | Formatted quota information | -| `{{ .FormatRateLimit }}` | Formatted rate limit information | - -### Catalogue Attributes (Within cart item) - -Accessible via `{{ $catalogue := $value.Catalogue }}` - -| Attribute | Description | -|-----------|-------------| -| `{{ .ID }}` | Catalogue ID | - -### Product Attributes (Within cart item) - -Accessible via `{{ range $product := $value.Products }}` - -| Attribute | Description | -|-----------|-------------| -| `{{ .ID }}` | Product ID | -| `{{ .Name }}` | Product name | -| `{{ .DisplayName }}` | Product display name | -| `{{ .Path }}` | Product path | -| `{{ .Description }}` | Product description | -| `{{ .Content }}` | Product content | -| `{{ .AuthType }}` | Product auth type | -| `{{ .DCREnabled }}` | true if product DCR enabled | -| `{{ .AuthTypes }}` | List of product auth types | - -### Auth Type Attributes (Within product) - -Accessible via `{{ range $auth_type := $product.AuthTypes }}` - -| Attribute | Description | -|-----------|-------------| -| `{{ .AuthType }}` | Auth type | - -### DCR Client Template Attributes (Within product) - -Accessible via `{{ range $template := $product.Templates }}` - -| Attribute | Description | -|-----------|-------------| -| `{{ .ID }}` | Template ID | -| `{{ .Name }}` | Template name | -| `{{ .GrantType }}` | Template grant type | -| `{{ .ResponseTypes }}` | Template response types | -| `{{ .TokenEndpointAuthMethod }}` | Template token endpoint auth method | -| `{{ .OktaAppType }}` | Template Okta app type | - -
    - Example Usage - -```html -

    Cart Checkout

    -{{ range $key, $value := .cart }} -
    -

    Auth Type: {{ $value.AuthType }}

    -

    Plan: {{ $value.Plan.Name }}

    - {{ range $product := $value.Products }} -
    -

    {{ $product.DisplayName }}

    -

    {{ $product.Description }}

    -

    Path: {{ $product.Path }}

    -
    - {{ end }} -
    -{{ end }} -

    Your Applications

    -{{ range $app := .apps }} -
    -

    {{ $app.Name }}

    -

    {{ $app.Description }}

    -
    -{{ end }} -{{ if .certs }} -

    MTLS Certificates

    -{{ range $cert := .certs }} -
    -

    ID: {{ $cert.ID }}

    -

    Name: {{ $cert.Name }}

    -
    -{{ end }} -{{ end }} -``` -
    - - -## Organization User Detail - -**Template Path**: `themes/default/views/user_detail.tmpl` - -This template is used to render the organization user detail page. - -### Available Objects - -- `{{ .errors }}`: Map of template errors (Key: category, Value: error message) -- `{{ .user }}`: The organization user object. - -### User Attributes - -Accessible via `{{ .user }}` - -| Attribute/Method | Description | -|-------------------|-------------| -| `{{ .ID }}` | User ID | -| `{{ .First }}` | User name | -| `{{ .Last }}` | User surname | -| `{{ .Email }}` | User email | -| `{{ .OrganisationID }}` | User organization ID | -| `{{ .DisplayName }}` | User complete name | -| `{{ .IdentityProvider }}` | User provider (Portal or Tyk Identity Broker) | -| `{{ .GetOrganisationID }}` | User's organization ID | -| `{{ .IsAdmin }}` | true if user is an admin | -| `{{ .IsOrgAdmin }}` | true if user is an organization admin | -| `{{ .DisplayRole }}` | User's role | - - -
    - Example Usage - -```html -

    User Details

    -{{ if .errors }} -{{ range $key, $errs := .errors }} - -{{ end }} -{{ end }} -

    Name: {{ .user.DisplayName }}

    -

    Email: {{ .user.Email }}

    -

    Role: {{ .user.DisplayRole }}

    -``` - -
    - -## Organization User Edit - -**Template Path**: `themes/default/views/user_edit.tmpl` - -This template is used to render the edit page for organization user. - -### Available Objects - -- `{{ .errors }}`: Map of template errors (Key: category, Value: error message) -- `{{ .roles }}`: List of possible roles -- `{{ .user }}`: The organization user object. - -### Role Attributes - -Accessible via `{{ range .roles }}` - -| Attribute | Description | -|-----------|-------------| -| `{{ .ID }}` | Role ID | -| `{{ .DisplayName }}` | Role display name | - -### User Attributes - -Accessible via `{{ .user }}` - -| Attribute/Method | Description | -|-------------------|-------------| -| `{{ .ID }}` | User ID | -| `{{ .First }}` | User name | -| `{{ .Last }}` | User surname | -| `{{ .Email }}` | User email | -| `{{ .OrganisationID }}` | User organization ID | -| `{{ .DisplayName }}` | User complete name | -| `{{ .IdentityProvider }}` | User provider (Portal or Tyk Identity Broker) | -| `{{ .GetOrganisationID }}` | User's organization ID | -| `{{ .IsAdmin }}` | true if user is an admin | -| `{{ .IsOrgAdmin }}` | true if user is an organization admin | -| `{{ .DisplayRole }}` | User's role | - -
    - Example Usage - -```html -
    - {{ if .error }} - - {{ end }} -

    Developer details

    -
    - - -
    -
    - - -
    -
    - - -
    - {{ if .roles }} -
    - - -
    - {{ end }} -
    - Cancel - -
    -
    -``` - -
    - -## Organization Users List - -**Template Path**: `themes/default/views/user_list.tmpl` - -This template is used to render the list of organization users. - -### Available Objects - -- `{{ .roles }}`: Map of available roles (Key: role, Value: role display name) - -#### Example Usage -```html - {{ index $roles $userInvite.Role }} -{{ end }} -``` - -## Product Detail - -**Template Path**: `themes/default/views/portal_product_detail.tmpl` - -This template is used to render the product detail page. - -### Available Objects - -- `{{ .product }}`: The selected product object -- `{{ .catalogues }}`: List of catalogue objects including the selected product -- `{{ .unique_plans }}`: List of plan objects available for the product -- `{{ .scopes }}`: Product scopes as an array of strings -- `{{ .posts }}`: List of related blog post objects -- `{{ .errors }}`: Map of template errors (Key: category, Value: error message) -- `{{ .added }}`: Boolean indicating if the product is added to the cart - -### Product Attributes - -Accessible via `{{ .product }}` - -| Attribute | Description | -|-----------|-------------| -| `{{ .ID }}` | Product ID | -| `{{ .Name }}` | Product name | -| `{{ .DisplayName }}` | Product display name | -| `{{ .Path }}` | Product path | -| `{{ .ReferenceID }}` | Product reference ID | -| `{{ .Description }}` | Product description | -| `{{ .AuthType }}` | Product auth type | -| `{{ .Logo.URL }}` | Product logo URL | -| `{{ .Feature }}` | true if the product is featured | -| `{{ .DCREnabled }}` | true if DCR is enabled | -| `{{ .ProviderID }}` | Provider ID | - -### API Details (Within product) - -Accessible via `{{ .product.APIDetails }}` - -| Attribute | Description | -|-----------|-------------| -| `{{ .Name }}` | API name | -| `{{ .Description }}` | API description | -| `{{ .APIType }}` | API type | -| `{{ .TargetURL }}` | API target URL | -| `{{ .ListenPath }}` | API listen path | -| `{{ .OASUrl }}` | API OAS URL | -| `{{ .Status }}` | "Active" if API status is active, otherwise "Inactive" | - -### Documentation (Within product) - -Accessible via `{{ .product.Docs }}` - -| Attribute | Description | -|-----------|-------------| -| `{{ .Title }}` | Document title | -| `{{ .ID }}` | Document identifier | -| `{{ .Content }}` | Document content | -| `{{ .MarkdownContent }}` | Markdown content | -| `{{ .MarkdownEnabled }}` | Boolean for Markdown enablement | - -### Catalogues - -Accessible via `{{ range .catalogues }}` - -| Attribute | Description | -|-----------|-------------| -| `{{ .Name }}` | Catalogue name | -| `{{ .VisibilityStatus }}` | Catalogue visibility status | - -### Plans - -Accessible via `{{ range .unique_plans }}` - -| Attribute | Description | -|-----------|-------------| -| `{{ .Name }}` | Plan name | -| `{{ .ID }}` | Plan ID | -| `{{ .DisplayName }}` | Plan display name | -| `{{ .Description }}` | Plan description | -| `{{ .AuthType }}` | Plan authentication type | -| `{{ .Rate }}` | Plan rate | -| `{{ .Per }}` | Plan rate per time unit | -| `{{ .QuotaMax }}` | Plan maximum quota | -| `{{ .QuotaRenewalRate }}` | Plan quota renewal rate | -| `{{ .AutoApproveAccessRequests }}` | Boolean for auto-approval of access requests | - -### Related Posts - -Accessible via `{{ range .posts }}` - -| Attribute | Description | -|-----------|-------------| -| `{{ .Title }}` | Post title | -| `{{ .Lede }}` | Post summary | -| `{{ .Content }}` | Post content | -| `{{ .MarkdownContent }}` | Markdown content | -| `{{ .MarkdownEnabled }}` | Boolean for Markdown enablement | -| `{{ .Path }}` | Post path | -| `{{ .HeaderImage.URL }}` | Header image URL | -| `{{ .BlogSiteID }}` | Blog site ID | -| `{{ .ProductID }}` | Associated product ID | -| `{{ .AuthorID }}` | Author ID | - -
    - Example Usage - -```html -
    -

    {{ .product.DisplayName }}

    - {{ .product.Name }} logo -

    {{ .product.Description }}

    -

    API Details

    - {{ range .product.APIDetails }} -

    {{ .Name }}

    -

    Status: {{ .Status }}

    -

    Target URL: {{ .TargetURL }}

    - {{ end }} -

    Documentation

    - {{ range .product.Docs }} -

    {{ .Title }}

    - {{ if .MarkdownEnabled }} - {{ .MarkdownContent | markdownify }} - {{ else }} - {{ .Content }} - {{ end }} - {{ end }} -

    Available in Catalogues

    -
      - {{ range .catalogues }} -
    • {{ .Name }} ({{ .VisibilityStatus }})
      • - {{ end }} -
    -

    Available Plans

    - {{ range .unique_plans }} -
    -

    {{ .DisplayName }}

    -

    {{ .Description }}

    -

    Rate: {{ .Rate }} per {{ .Per }}

    -

    Quota: {{ .QuotaMax }}

    -
    - {{ end }} -

    Related Posts

    - {{ range .posts }} -
    -

    {{ .Title }}

    - {{ .Title }} -

    {{ .Lede }}

    -
    - {{ end }} -
    -``` - -
    - -## Product OAS Documentation - -**Template Paths**: -- `themes/default/views/product_doc_stoplight_spec.tmpl` -- `themes/default/views/product_doc_redoc.tmpl` - -These templates are used to render the OpenAPI Specification (OAS) documentation for a product. The Stoplight Spec and -ReDoc versions are available. - -### Available Attributes - -| Attribute | Description | -|-----------|-------------| -| `{{ .Name }}` | Product name | -| `{{ .Description }}` | Product description | -| `{{ .Path }}` | Product path | -| `{{ .Url }}` | OAS document URL | - -
    - Example Usage - -```html -
    -
    -
    -

    - {{ .Name }} -

    -

    - {{ .Description }} -

    -
    -
    -
    - -
    -
    -``` - -
    - -# Global Helper Functions - -This section provides a detailed overview of the global helper functions available in the Tyk Enterprise Developer -Portal templates. These functions are accessible across the public and private templates and allow you to perform -various operations, retrieve specific data, and create dynamic content within your templates. - -## Available Functions - -- [CanCreateOrganisation](#cancreateorganisation) -- [Clients](#clients) -- [Current User](#currentuser) -- [FeaturedProducts](#featuredproducts) -- [FilterUserInvites](#filteruserinvites) -- [FormatTime](#formattime) -- [GetCart](#getcart) -- [GetCatalogueList](#getcataloguelist) -- [GetCataloguesForProduct](#getcataloguesforproduct) -- [GetClientDescription](#getclientdescription) -- [GetClientName](#getclientname) -- [GetMenus](#getmenus) -- [GetProducts](#getproducts) -- [IsPortalDisabled](#isportaldisabled) -- [IsPortalPrivate](#isportalprivate) -- [ProductDocRenderer](#productdocrenderer) -- [ProviderUpstreamURL](#providerupstreamurl) -- [SplitStrings](#splitstrings) -- [TruncateString](#truncatestring) -- [TypeOfCredential](#typeofcredential) - -### CanCreateOrganisation - -Returns true if user can create an organization. - -#### Example Usage -``` -{{ if CanCreateOrganisation req }} - ... -{{ end }} -``` - -### Clients - -Returns the list of applications for the current user. Expects the request as argument. - -#### Client Attributes - -Accessible via `{{ range $client := Clients req }}` - -| Attribute | Description | -|-----------|-------------| -| `{{ $client.ID }}` | Client ID | -| `{{ $client.Name }}` | Client name | -| `{{ $client.Description }}` | Client description | -| `{{ $client.RedirectURLs }}` | Client redirect URLs | -| `{{ $client.Credentials }}` | Array of client credentials | -| `{{ $client.AccessRequests }}` | Array of client access requests | - -#### Credential Attributes (Within client) - -Accessible via `{{ range $cred := $client.Credentials }}` - -| Attribute | Description | -|-----------|-------------| -| `{{ $cred.ID }}` | Credential ID | -| `{{ $cred.Credential }}` | Credential | -| `{{ $cred.CredentialHash }}` | Credential hash | -| `{{ $cred.OAuthClientID }}` | OAuth client ID | -| `{{ $cred.OAuthClientSecret }}` | OAuth client secret | -| `{{ $cred.Expires }}` | Credential expiration | -| `{{ $cred.AccessRequestID }}` | Access request ID associated with the credential | - -#### Access Request Attributes (Within client) - -Accessible via `{{ range $acreq := $client.AccessRequests }}` - -| Attribute | Description | -|-----------|-------------| -| `{{ $acreq.ID }}` | Access request ID | -| `{{ $acreq.Status }}` | Access request status | -| `{{ $acreq.UserID }}` | User ID associated with access request | -| `{{ $acreq.AuthType }}` | Access request auth type | -| `{{ $acreq.DCREnabled }}` | true if access request DCR enabled | -| `{{ $acreq.ProvisionImmediately }}` | true if provisioned immediately is enabled | -| `{{ $acreq.CatalogueID }}` | Catalogue ID | - -#### Product Attributes (Within access request) - -Accessible via `{{ range $product := $acreq.Products }}` - -| Attribute | Description | -|-----------|-------------| -| `{{ $product.ID }}` | Product ID | -| `{{ $product.Name }}` | Product name | -| `{{ $product.DisplayName }}` | Product display name | -| `{{ $product.Path }}` | Product path | -| `{{ $product.Description }}` | Product description | -| `{{ $product.Content }}` | Product content | -| `{{ $product.AuthType }}` | Product auth type | -| `{{ $product.DCREnabled }}` | true if product DCR enabled | - -#### Plan Attributes (Within access request) - -Accessible via `{{ $acreq.Plan }}` - -| Attribute | Description | -|-----------|-------------| -| `{{ .Name }}` | Plan name | -| `{{ .DisplayName }}` | Plan display name | -| `{{ .Description }}` | Plan description | -| `{{ .AuthType }}` | Plan auth type | -| `{{ .Rate }}` | Plan rate | -| `{{ .Per }}` | Plan period | -| `{{ .QuotaMax }}` | Plan quota maximum | -| `{{ .QuotaRenewalRate }}` | Plan quota renewal rate | -| `{{ .AutoApproveAccessRequests }}` | true if auto-approve access requests is enabled | - -#### Example Usage -```html -{{ range $client := Clients req }} -
    -

    Client: {{ $client.Name }}

    - {{ range $acreq := $client.AccessRequests }} -

    Products:

    -
      - {{ range $product := $acreq.Products }} -
    • - {{ $product.Name }} - {{ $product.Description }} -
      • - {{ end }} -
    -

    Plan:

    -

    Name: {{ $acreq.Plan.Name }}

    -

    Rate: {{ $acreq.Plan.Rate }} per {{ $acreq.Plan.Per }}

    -

    Quota Max: {{ $acreq.Plan.QuotaMax }}

    - {{ end }} -
    -{{ end }} -``` - -### CurrentUser - -The `CurrentUser` function returns the current user object if a user is logged in. It expects the request as an argument. - -### User Attributes - -Accessible via `{{ $user := CurrentUser req }}` - -| Attribute/Method | Description | -|-------------------|-------------| -| `{{ $user.ID }}` | User ID | -| `{{ $user.First }}` | User name | -| `{{ $user.Last }}` | User surname | -| `{{ $user.Email }}` | User email | -| `{{ $user.OrganisationID }}` | User organization ID | -| `{{ $user.DisplayName }}` | User complete name | -| `{{ $user.IdentityProvider }}` | User provider (Portal or Tyk Identity Broker) | -| `{{ $user.GetOrganisationID }}` | User's organization ID | -| `{{ $user.IsAdmin }}` | true if user is an admin | -| `{{ $user.IsOrgAdmin }}` | true if user is an organization admin | -| `{{ $user.DisplayRole }}` | User's role | - -#### Example Usage -```html -{{ $user := CurrentUser req }} -{{ if $user }} - -{{ else }} -

    Please log in to view your account information.

    -{{ end }} -``` - -### FeaturedProducts - -Returns a list of featured products. - -#### Product Attributes - -Accessible via `{{ range FeaturedProducts }}` - -| Attribute | Description | -|-----------|-------------| -| `{{ .ID }}` | Product ID | -| `{{ .Name }}` | Product name | -| `{{ .DisplayName }}` | Product display name | -| `{{ .Path }}` | Product path | -| `{{ .ReferenceID }}` | Product reference ID | -| `{{ .Description }}` | Product description | -| `{{ .AuthType }}` | Product auth type | -| `{{ .Scopes }}` | Product scopes | -| `{{ .Logo.URL }}` | Product logo URL | -| `{{ .Feature }}` | true if the product is featured | -| `{{ .DCREnabled }}` | true if DCR is enabled | -| `{{ .ProviderID }}` | Provider ID | -| `{{ .APIDetails }}` | Array of API details associated with the product | -| `{{ .Catalogues }}` | Array of catalogues associated with the product | - -#### API Details Attributes (Within product) - -Accessible via `{{ range .APIDetails }}` - -| Attribute | Description | -|-----------|-------------| -| `{{ .Name }}` | API name | -| `{{ .Description }}` | API description | -| `{{ .APIType }}` | API type | -| `{{ .TargetURL }}` | API target URL | -| `{{ .ListenPath }}` | API listen path | -| `{{ .OASUrl }}` | API OAS URL | -| `{{ .Status }}` | "Active" if API status is active, otherwise "Inactive" | - -#### Catalogue Attributes (Within product) - -Accessible via `{{ range .Catalogues }}` - -| Attribute | Description | -|-----------|-------------| -| `{{ .Name }}` | Catalogue name | -| `{{ .VisibilityStatus }}` | Catalogue visibility status | - -
    - Example Usage - -```html -{{ $featured_products := FeaturedProducts }} -

    Featured API Products

    -

    Explore our highlighted API offerings

    -
    - {{ range $featured_products }} -
    - {{ if .Logo }} - {{ .Name }} logo - {{ end }} -
    - {{ .AuthType }} -

    {{ .Name }}

    -

    {{ .Description }}

    -
    -
    - More Info -
    - {{ range .APIDetails }} - {{ if or (gt (.OASDocument.Base.Url | trim | length) 0) (gt (.OASUrl | trim | length) 0) }} - - {{ .Name }} - - {{ end }} - {{ end }} -
    -
    -
    - {{ end }} -
    -``` - -
    - -### FilterUserInvites - -Returns a list of users that were invited to the current user's organization, if the user became an organization. -Expects the request as a parameter. - -#### User Attributes - -Accessible via `{{ range $invite := FilterUserInvites req }}` - -| Attribute | Description | -|-----------|-------------| -| `{{ $invite.ID }}` | User ID | -| `{{ $invite.Email }}` | User email | -| `{{ $invite.First }}` | User first name | -| `{{ $invite.Last }}` | User last name | -| `{{ $invite.Role }}` | User role | -| `{{ $invite.JoinedAt }}` | User joined at time | -| `{{ $invite.Joined }}` | Whether the user has joined | -| `{{ $invite.Uactive }}` | Whether the user is active | - -
    - Example Usage - -```html -{{ $userInvites := FilterUserInvites req }} -{{ if $userInvites }} -

    Invited Users

    - - - - - - - - - - - {{ range $invite := $userInvites }} - - - - - - - {{ end }} - -
    NameEmailRoleStatus
    {{ $invite.First }} {{ $invite.Last }}{{ $invite.Email }}{{ $invite.Role }} - {{ if $invite.Joined }} - Joined - {{ else if $invite.Uactive }} - Pending - {{ else }} - Inactive - {{ end }} -
    -{{ else }} -

    No pending invitations.

    -{{ end }} -``` - -
    - -### FormatTime - -Formats a given time with a given format. - -#### Example Usage -```gotemplate -{{ $user := CurrentUser req }} -{{ if $user}} -{{$time := FormatTime $user.CreatedAt "2 Jan, 2006 at 3:04:00 PM (MST)"}} - -... -{{end}} -``` - -### GetCart - -Returns a map with the cart items for a given user ID. Expects the user ID as an argument. This function is useful for -retrieving and displaying the contents of a user's cart, including detailed information about the products, their -authentication types, and associated templates. - -#### Cart Item Attributes - -Accessible via `{{ range $key, $value := GetCart $user.ID }}` - -| Attribute | Description | -|-----------|-------------| -| `{{ $value.AuthType }}` | Cart item auth type | -| `{{ $value.Catalogue }}` | Cart item catalogue | -| `{{ $value.DCREnabled }}` | true if cart order consists of DCR products | -| `{{ $value.Plan }}` | Cart item plan | -| `{{ $value.Products }}` | Cart item array of products | - -### Plan Attributes (Within cart item) - -Accessible via `{{ $plan := $value.Plan }}` - -| Attribute | Description | -|-----------|-------------| -| `{{ .ID }}` | Plan ID | -| `{{ .PlanName }}` | Plan name | -| `{{ .FormatQuota }}` | Formatted quota information | -| `{{ .FormatRateLimit }}` | Formatted rate limit information | - -### Catalogue Attributes (Within cart item) - -Accessible via `{{ $catalogue := $value.Catalogue }}` - -| Attribute | Description | -|-----------|-------------| -| `{{ .ID }}` | Catalogue ID | - -#### Product Attributes (Within cart item) - -Accessible via `{{ range $product := $value.Products }}` - -| Attribute | Description | -|-----------|-------------| -| `{{ .ID }}` | Product ID | -| `{{ .Name }}` | Product name | -| `{{ .DisplayName }}` | Product display name | -| `{{ .Path }}` | Product path | -| `{{ .Description }}` | Product description | -| `{{ .Content }}` | Product content | -| `{{ .AuthType }}` | Product auth type | -| `{{ .DCREnabled }}` | true if product DCR enabled | -| `{{ .AuthTypes }}` | List of product auth types | - -#### DCR Client Template Attributes (Within product) - -Accessible via `{{ range $template := $product.Templates }}` - -| Attribute | Description | -|-----------|-------------| -| `{{ .ID }}` | Template ID | -| `{{ .Name }}` | Template name | -| `{{ .GrantType }}` | Template grant type | -| `{{ .ResponseTypes }}` | Template response types | -| `{{ .TokenEndpointAuthMethod }}` | Template token endpoint auth method | -| `{{ .OktaAppType }}` | Template Okta app type | - -#### Example Usage -```html -{{ $user := CurrentUser req }} -{{ if $user }} - {{ $cart := GetCart $user.ID }} - {{ if $cart }} -

    Your Cart

    - {{ range $key, $value := $cart }} -
    -

    {{ $value.Catalogue.Name }}

    -

    Auth Type: {{ $value.AuthType }}

    - {{ range $product := $value.Products }} -
    -

    {{ $product.DisplayName }}

    -

    {{ $product.Description }}

    -
    - {{ end }} -
    - {{ end }} - {{ else }} -

    Your cart is empty.

    - {{ end }} -{{ end }} -``` - -### GetCatalogueList - -Returns a list of catalogue names. Expects the request as parameter. - -#### Example Usage -```gotemplate -{{ range $key, $value := GetCatalogueList req }} - -{{ end }} -``` - -### GetCataloguesForProduct - -Returns a list of products for a given user and product ID. Expects the request, a user and a product ID as parameters. - -#### Catalogue Attributes - -Accessible via `{{ range GetCataloguesForProduct req $user $product.ID }}` - -| Attribute | Description | -|-----------|-------------| -| `{{ .VisibilityStatus }}` | Catalogue visibility status | -| `{{ .Name }}` | Catalogue name | - -#### Example Usage -```html -{{ $thisProduct := .product }} -{{ $user := CurrentUser req }} -{{ $catalogues_for_product := GetCataloguesForProduct req $user $thisProduct.ID }} -

    Catalogues for {{ $thisProduct.Name }}

    -
      - {{ range $catalogues_for_product }} -
    • - {{ .Name }} - (Visibility: {{ .VisibilityStatus }}) -
      • - {{ end }} -
    -``` - -### GetClientDescription - -Returns an application description given a credential ID. - -#### Example Usage -``` -{{ range $app.Credentials }} -... -{{ GetClientDescription .ID}} -{{end}} -``` - -### GetClientName - -Returns an application name given a credential ID. - -#### Example Usage -``` -{{ range $app.Credentials }} -... -{{ GetClientName .ID}} -{{end}} -``` - -### GetMenus - -Returns a map of all [menus]({{< ref "/tyk-stack/tyk-developer-portal/enterprise-developer-portal/customise-enterprise-portal/full-customisation/menus-customisation" >}}). - -#### Example Usage -```html -{{ if GetMenus.Primary }} - {{ range GetMenus.Primary.Children }} - {{ range .Children }} -
  • - {{ .Tag }} -
    • - {{ end }} - {{ end }} -{{ end }} -``` - -### GetProducts - -Returns the list of products for the current user. Expects the request as an argument. - -#### Product Attributes - -Accessible via `{{ range $product := GetProducts req }}` - -| Attribute | Description | -|-----------|-------------| -| `{{ $product.ID }}` | Product ID | -| `{{ $product.Name }}` | Product name | -| `{{ $product.DisplayName }}` | Product display name | -| `{{ $product.Path }}` | Product path | -| `{{ $product.ReferenceID }}` | Product reference ID | -| `{{ $product.Description }}` | Product description | -| `{{ $product.AuthType }}` | Product auth type | -| `{{ $product.Scopes }}` | Product scopes | -| `{{ $product.Logo.URL }}` | Product logo URL | -| `{{ $product.Feature }}` | true if the product is featured | -| `{{ $product.DCREnabled }}` | true if DCR is enabled | -| `{{ $product.ProviderID }}` | Provider ID | -| `{{ $product.APIDetails }}` | Array of API details associated with the product | -| `{{ $product.Catalogues }}` | Array of catalogues associated with the product | - -#### API Details Attributes (Within product) - -Accessible via `{{ range $api := $product.APIDetails }}` - -| Attribute | Description | -|-----------|-------------| -| `{{ $api.Name }}` | API name | -| `{{ $api.Description }}` | API description | -| `{{ $api.APIType }}` | API type | -| `{{ $api.TargetURL }}` | API target URL | -| `{{ $api.ListenPath }}` | API listen path | -| `{{ $api.OASUrl }}` | API OAS URL | -| `{{ $api.Status }}` | "Active" if API status is active, otherwise "Inactive" | - -#### Catalogue Attributes (Within product) - -Accessible via `{{ range $catalogue := $product.Catalogues }}` - -| Attribute | Description | -|-----------|-------------| -| `{{ $catalogue.Name }}` | Catalogue name | -| `{{ $catalogue.VisibilityStatus }}` | Catalogue visibility status | - -
    - Example Usage - -```html -{{ range GetProducts req }} -
    -
    - {{ if .Logo.URL }} - - {{ end }} -
    -
    -
    - {{ .AuthType }} -
    -

    {{ .ProductName }}

    -
    - {{ if .Description }} -

    {{ .Description }}

    - {{ end }} -
    -
    -
    - More Info -
    -
    -
    -
    -{{ end }} -``` - -
    - -### IsPortalDisabled - -Returns true (exception: for admins is always enabled) if portal visibility was set to hidden. Expects the request as -parameter. - -#### Example Usage -``` -{{ $portalDisabled := IsPortalDisabled req }} -``` - -### IsPortalPrivate - -Returns true (exception: for admins is always enabled) if portal visibility was set to private. Expects the request as -parameter. - -#### Example Usage -``` -{{ $portalPrivate := IsPortalPrivate req }} -``` - -### ProductDocRenderer - -Returns the configured product OAS renderer (redoc or stoplight). - -#### Example Usage -``` -{{ $oas_template := ProductDocRenderer }} -``` - -### ProviderUpstreamURL - -Returns the provider upstream URL for a given providerID. Expects the request and a provider ID as parameters. - -#### Example Usage -``` -{{ $upstreamURL := ProviderUpstreamURL req $thisProduct.ProviderID }} -``` - -### SplitStrings - -Splits a given string with given separator and returns a slice of split strings. - -#### Example Usage -``` -{{ range $app.Credentials }} -... -{{ range SplitStrings .GrantType "," }} -... -{{ end }} -{{ end }} -``` - -### TruncateString - -Truncates a given string to a given length, returning the truncated string followed by three dots (…). - -#### Example Usage -``` -{{ TruncateString $api.Description 60 }} -``` - -### TypeOfCredential - -Returns the credential type ("oAuth2.0" or "authToken") given the credential. - -#### Example Usage -``` -{{ range $app.Credentials }} -... -{{ if eq (TypeOfCredential . ) "oAuth2.0" }} -... -{{ end }} -{{end}} -``` - - -# Email Templates - -This section provides a detailed overview of the email template data available in the Tyk Enterprise Developer Portal. -The Tyk Enterprise Developer Portal uses a variety of email templates for different purposes, such as user registration -and access request status or organization status updates. Each template has access to specific data or functions relevant -to its purpose. - -It's important to note that while email templates can include template data or specific template functions, they do not -have access to the global helper functions available in other portal templates. - -Please refer to [email workflow]({{< ref "/tyk-stack/tyk-developer-portal/enterprise-developer-portal/customise-enterprise-portal/full-customisation/email-customization" >}}) -for additional detail on email notifications sent by the portal. - - -## Available Email Templates - -- [Access Request Approve/Reject](#access-request-approvereject) -- [Access Request Submitted](#access-request-submitted) -- [Activate and Deactivate](#activate-and-deactivate) -- [New User Request](#new-user-request) -- [Organization Approve](#organization-approve) -- [Organization Reject](#organization-reject) -- [Organization Request](#organization-request) -- [Reset Password](#reset-password) -- [Targeted Invite](#targeted-invite) -- [Welcome User](#welcome-user) - -### Access Request Approve/Reject - -**Template Paths**: -- `themes/default/mailers/approve.tmpl` -- `themes/default/mailers/reject.tmpl` - -These templates are used for sending notifications to users when their access requests are approved or rejected. - -#### Available Objects - -There's no data sent to these templates. - -#### Example Usage -``` -Hi, -The API Credentials you provisioned have been rejected. -Thanks, -The Team -``` - -### Access Request Submitted - -**Template Path**: `themes/default/mailers/submitted.tmpl` - -This template is used for notifying administrators about pending access requests. - -#### Available Objects - -- `{{ .requests }}`: Returns the list of access requests pending approval. - -#### Access Request Attributes - -Accessible via `{{ range .requests }}` - -| Attribute | Description | -|-----------|-------------| -| `{{ .PlanID }}` | Plan ID associated with access request | -| `{{ .Status }}` | Request status | -| `{{ .AuthType }}` | Request authentication type | -| `{{ .UserID }}` | User ID associated with the request | -| `{{ .ClientID }}` | Client ID associated with the request | -| `{{ .DCREnabled }}` | Indicates if DCR (Dynamic Client Registration) is enabled for the request | -| `{{ .ProvisionImmediately }}` | Indicates if provisioning is immediate for the request | -| `{{ .CatalogueID }}` | Catalogue ID associated with the request | - -#### Product Attributes (within Access Request) - -Accessible via `{{ range $product := $acreq.Products }}` - -| Attribute | Description | -|-----------|-------------| -| `{{ $product.ID }}` | Product ID | -| `{{ $product.Name }}` | Product name | -| `{{ $product.DisplayName }}` | Product display name | -| `{{ $product.Description }}` | Product description | -| `{{ $product.AuthType }}` | Product authentication type | -| `{{ $product.DCREnabled }}` | Indicates if DCR (Dynamic Client Registration) is enabled for the product | - -#### Example Usage -```html -

    A new Access request has been submitted. Please log in to the administration dashboard to view the request.

    -
      - {{ range $acreq := .requests }} -
    • - Status: {{ $acreq.Status }}
      - User ID: {{ $acreq.UserID }}
      - Products: -
        - {{ range $product := $acreq.Products }} -
      • {{ $product.DisplayName }} ({{ $product.AuthType }})
        • - {{ end }} -
      -
      • - {{ end }} -
    -``` - - -### Activate and Deactivate - -**Template Paths**: -- `themes/default/mailers/activate.tmpl` -- `themes/default/mailers/deactivate.tmpl` - -These templates are used for sending activation and deactivation notifications to users. - -#### Available Objects - -- `{{ .name }}`: Returns the user's full name. - -#### Example Usage -``` -Hi, {{.name}}
    -Your account has been activated. -``` - -### New User Request - -**Template Path**: `themes/default/mailers/newuser.tmpl` - -This template is used for notifying administrators about new user registration requests pending activation. - -#### Available Objects - -- `{{ .user }}`: Returns the new user pending activation. - -### User Attributes - -Accessible via `{{ .user }}` - -| Attribute/Method | Description | -|-------------------|-------------| -| `{{ .ID }}` | User ID | -| `{{ .First }}` | User name | -| `{{ .Last }}` | User surname | -| `{{ .Email }}` | User email | -| `{{ .OrganisationID }}` | User organization ID | -| `{{ .DisplayName }}` | User complete name | -| `{{ .IdentityProvider }}` | User provider (Portal or Tyk Identity Broker) | -| `{{ .GetOrganisationID }}` | User's organization ID | -| `{{ .IsAdmin }}` | true if user is an admin | -| `{{ .IsOrgAdmin }}` | true if user is an organization admin | -| `{{ .DisplayRole }}` | User's role | -| `{{ .Organisation.Name }}` | Organization name | -| `{{ .Teams }}` | Array of user teams | -| `{{ .Teams.ID }}` | Team ID | -| `{{ .Teams.Name }}` | Team name | -| `{{ .Teams.Default }}` | Indicates if the team is the default team (true/false) | - -#### Example Usage -``` -

    There is a new user request pending. Please approve it from the admin console.

    -

    - Id: {{ .user.ID }}
    - User: {{ .user.DisplayName }} ({{ .user.Email }})
    - Role: {{ .user.Role }}
    - {{ if gt .user.OrganisationID 0 }} - Organisation: {{ .user.Organisation.Name }}
    - {{ else }} - Organisation: Administrators' organisation
    - {{ end }} - {{ if gt (len .user.Teams) 0 }} - Teams:
    -

      - {{ range .user.Teams }} -
    • {{ .Name }}
      • - {{ end }} -
    - {{ else }} - Teams: none - {{ end }} -

    -``` - -### Organization Approve - -**Template Path**: `themes/default/mailers/organisation_request.tmpl` - -This template is used for notifying users that their organization creation request has been approved. - -#### Available Objects - -- `{{ site }}`: Returns the application host. - -#### Example Usage -``` -Hello, -The organization registration request has been approved. You can now manage your organization in your dashboard here: https://{{.site}}/portal/private/dashboard -Thanks, -The team -``` - -### Organization Reject - -**Template Path**: `themes/default/mailers/organisation_reject.tmpl` - -This template is used for notifying users that their organization creation request has been rejected. - -#### Available Objects - -There's no data sent to this template. - -#### Example Usage -``` -Hello, -The organization registration request has been rejected. -Thanks, -The team -``` - -### Organization Request - -**Template Path**: `themes/default/mailers/organisation_request.tmpl` - -This template is used for notifying administrators about new organization creation requests. - -#### Available Objects - -- `{{ .user }}`: Returns the user who made the request. -- `{{ .organisationName }}`: Returns the new organization name. - -### User Attributes - -Accessible via `{{ .user }}` - -| Attribute/Method | Description | -|-------------------|-------------| -| `{{ .ID }}` | User ID | -| `{{ .First }}` | User name | -| `{{ .Last }}` | User surname | -| `{{ .Email }}` | User email | -| `{{ .OrganisationID }}` | User organization ID | -| `{{ .DisplayName }}` | User complete name | -| `{{ .IdentityProvider }}` | User provider (Portal or Tyk Identity Broker) | -| `{{ .GetOrganisationID }}` | User's organization ID | -| `{{ .IsAdmin }}` | true if user is an admin | -| `{{ .IsOrgAdmin }}` | true if user is an organization admin | -| `{{ .DisplayRole }}` | User's role | - -#### Example Usage -``` -There is a new organization registration request pending. Please approve it from the admin console. -The organization name: {{ .organisationName }}. -The user: {{ .user.DisplayName }} ({{ .user.Email }}). -``` - -### Reset Password - -**Template Path**: `themes/default/mailers/auth/reset_password.tmpl` - -This template is used for sending password reset emails to users. - -#### Available Functions - -- `{{ current_user }}`: Returns the current user object. -- `{{ reset_password_url }}`: Returns the URL with the token for setting the password. - -#### User Attributes - -Accessible via `{{ current_user }}` - -| Attribute/Method | Description | -|-------------------|-------------| -| `{{ .ID }}` | User ID | -| `{{ .First }}` | User name | -| `{{ .Last }}` | User surname | -| `{{ .Email }}` | User email | -| `{{ .Role }}` | User role | -| `{{ .OrganisationID }}` | User organization ID | -| `{{ .DisplayName }}` | User complete name | -| `{{ .IdentityProvider }}` | User provider (Portal or Tyk Identity Broker) | -| `{{ .GetOrganisationID }}` | User's organization ID | -| `{{ .IsAdmin }}` | true if user is an admin | -| `{{ .IsOrgAdmin }}` | true if user is an organization admin | -| `{{ .DisplayRole }}` | User's role | - -#### Example Usage -``` -{{ $user := current_user}} -

    Hello {{ $user.DisplayName }},

    -

    Someone has requested a link to change your password. You can do this through the link below.

    -

    {{reset_password_url}}

    -

    If you didn't request this, please ignore this email.

    -

    Your password won't change until you access the link above and create a new one.

    -``` - -### Targeted Invite - -**Template Path**: `themes/default/mailers/auth/targeted_invite.tmpl` - -This template is used for sending targeted invitations to users. - -#### Available Functions - -- `{{ user }}`: Returns the targeted user object. -- `{{ team }}`: Returns the team name to which the user is being invited. -- `{{ invite_url }}`: Returns the URL with the token for setting the password. - -#### User Attributes - -Accessible via `{{ user }}` - -| Attribute/Method | Description | -|-------------------|-------------| -| `{{ .ID }}` | User ID | -| `{{ .First }}` | User name | -| `{{ .Last }}` | User surname | -| `{{ .Email }}` | User email | -| `{{ .Role }}` | User role | -| `{{ .OrganisationID }}` | User organization ID | -| `{{ .DisplayName }}` | User complete name | -| `{{ .IdentityProvider }}` | User provider (Portal or Tyk Identity Broker) | -| `{{ .GetOrganisationID }}` | User's organization ID | -| `{{ .IsAdmin }}` | true if user is an admin | -| `{{ .IsOrgAdmin }}` | true if user is an organization admin | -| `{{ .DisplayRole }}` | User's role | - -#### Example Usage -```html -{{ $u := user }} -Hi, {{ $u.DisplayName }}
    -

    Someone is inviting you to join {{ if $u.IsAdmin }}as an Administrator{{ else }}the {{ team }} team{{end }}. You can do this through the link below.

    -

    {{ invite_url }}

    -

    If you didn't request this, please ignore this email.

    -``` - -### Welcome User - -**Template Paths**: -- `themes/default/mailers/welcome_admin.tmpl` -- `themes/default/mailers/welcome_dev.tmpl` - -These templates are used for sending welcome emails to new users, with separate templates for administrators and developers. - -#### Available Objects - -- `{{ .user }}`: Returns the user who made the request. Refer to the CurrentUser section for accessible attributes and methods. - -### User Attributes - -Accessible via `{{ .user }}` - -| Attribute/Method | Description | -|-------------------|-------------| -| `{{ .ID }}` | User ID | -| `{{ .First }}` | User name | -| `{{ .Last }}` | User surname | -| `{{ .Email }}` | User email | -| `{{ .OrganisationID }}` | User organization ID | -| `{{ .DisplayName }}` | User complete name | -| `{{ .IdentityProvider }}` | User provider (Portal or Tyk Identity Broker) | -| `{{ .GetOrganisationID }}` | User's organization ID | -| `{{ .IsAdmin }}` | true if user is an admin | -| `{{ .IsOrgAdmin }}` | true if user is an organization admin | -| `{{ .DisplayRole }}` | User's role | -| `{{ .Organisation.Name }}` | organization name | -| `{{ .Teams }}` | Array of user teams | -| `{{ .Teams.ID }}` | Team ID | -| `{{ .Teams.Name }}` | Team name | -| `{{ .Teams.Default }}` | Indicates if the team is the default team (true/false) | - -
    - Example Usage - -```html -

    Welcome to Tyk Enterprise Developer Portal

    -

    Hello {{ .user.DisplayName }},

    -

    Your account has been created for the {{ .user.Organisation.Name }} organisation.

    -

    Your assigned teams:

    -
      - {{ range .user.Teams }} -
    • {{ .Name }}{{ if .Default }} (Default){{ end }}
      • - {{ end }} -
    -

    We're excited to have you on board!

    -``` - -
    \ No newline at end of file diff --git a/tyk-docs/content/tyk-stack/tyk-developer-portal/enterprise-developer-portal/customise-enterprise-portal/quick-customisation.md b/tyk-docs/content/tyk-stack/tyk-developer-portal/enterprise-developer-portal/customise-enterprise-portal/quick-customisation.md deleted file mode 100644 index fb9892262d..0000000000 --- a/tyk-docs/content/tyk-stack/tyk-developer-portal/enterprise-developer-portal/customise-enterprise-portal/quick-customisation.md +++ /dev/null @@ -1,109 +0,0 @@ ---- -title: "Quick Customization" -date: 2022-02-08 -tags: [""] -description: "" -menu: - main: - parent: "Customize the Enterprise Portal" -weight: 1 ---- - -{{< note success >}} -**Tyk Enterprise Developer Portal** - -If you are interested in getting access contact us at [support@tyk.io]() - -{{< /note >}} - -## Introduction - -In this section we will explain how to apply your branding (styling - CSS) on the portal elements with your own colors and logo within minutes. - -## Prerequisites - -- A Tyk Self-Managed [installation]({{< ref "tyk-self-managed#installation-options-for-tyk-self-managed" >}}) -- A login for the portal admin app -- Access to your Tyk portal file system - -## Step by step instructions - -### Part 1 - Changing the portal logo - -1. Access the file directory for the Developer portal -2. The default logo is located in `/themes/default/assets/images/` and is called `dev-portal-logo.svg`. -3. Replace the default image with your own, keeping the same file name and in `.svg` format, ensuring that `xmlns="http://www.w3.org/2000/svg"` is included within your `` tag. - -{{< note success >}} -**Note** - -If you want to use different naming, path reference or extension, the code is `` and is found on line 6 from the `/themes/default/partials/footer.tmpl` template. -{{< /note >}} - -### Part 2 - Changing brand colors - -Let’s now explain how to manage borders and change the colors of buttons, texts and backgrounds. The file we’ll be looking at is `/themes/default/assets/stylesheets/main.css` which contains some CSS variables that are used throughout the app. Let’s take a closer look. -You can apply some changes in the portal based on your preferences. For example, you can change the navigation background color, the text color and the different button theme colors. Furthermore, you can change table border color and radius. - -If you want to change the navigation background color you need to edit the variable called `--tdp-nav-bg-color` Similarly other variables as you can see where/how each one is used: - -{{< note success >}} -**Note** - -`tdp` stands for Tyk Developer Portal - -{{< /note >}} - -#### Background colors - -{{< img src="/img/dashboard/portal-management/enterprise-portal/background-colors.png" alt="Background Colour settings Tyk Enterprise Portal" >}} - -- `--tdp-nav-bg-color` navigation background color -- `--tdp-body-bg-color` App background color - -#### Text colors - -{{< img src="/img/dashboard/portal-management/enterprise-portal/text-colors.png" alt="Text Colour settings Tyk Enterprise Portal" >}} - -- `--tdp-text-color` default text color -- `--tdp-link-color` links (anchor tags) -- `--tdp-nav-link-color` navigation links - -#### Borders - -{{< img src="/img/dashboard/portal-management/enterprise-portal/borders.png" alt="Border Colour settings Tyk Enterprise Portal" >}} - -- `--tdp-card-border-radius` Card component -- `--tdp-border-color-on-error` input color if there’s an error -- `--tdp-table-border-color` table -- `--tdp-border-radius` radius -- `--tdp-primary-border form` elements (such as `` and `` and `