[TT-7902] Detailed Tracing #5294

mativm02 · 2023-07-07T13:55:41Z

Description

Adding detailed_tracing field to API Definition. If this is set to true, then each OpenTelemetry trace will propagate the context and multiple spans will be shown. Otherwise, only the main span and a span showing the request to the upstream service will be present.

Related Issue

https://tyktech.atlassian.net/browse/TT-7902?atlOrigin=eyJpIjoiMjgwMDY3YTQ5Nzk0NDAyODlkODUxOTUzMzc5NTZhZDgiLCJwIjoiaiJ9

Motivation and Context

How This Has Been Tested

Screenshots (if appropriate)

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)
Refactoring or add test (improvements in base code or adds test coverage to functionality)

Checklist

I ensured that the documentation is up to date
I explained why this PR updates go.mod in detail with reasoning why it's required
I would like a code coverage CI quality gate exception and have explained why

Tyk-ITS · 2023-07-07T14:15:05Z

API tests result: failure 🚫
Branch used: refs/pull/5294/merge
Commit:
Triggered by: pull_request (@mativm02)
Execution page

Tyk-ITS · 2023-07-07T14:52:49Z

API tests result: failure 🚫
Branch used: refs/pull/5294/merge
Commit: 10c54c7
Triggered by: pull_request (@mativm02)
Execution page

github-actions · 2023-07-07T15:05:00Z

💥 CI tests failed 🙈

git-state

all ok

Please look at the run or in the Checks tab.

Tyk-ITS · 2023-07-07T15:17:13Z

API tests result: failure 🚫
Branch used: refs/pull/5294/merge
Commit: 45f08e1
Triggered by: pull_request (@mativm02)
Execution page

github-actions · 2023-07-10T12:51:27Z

💥 CI tests failed 🙈

git-state

all ok

Please look at the run or in the Checks tab.

github-actions · 2023-07-10T12:51:55Z

💥 CI tests failed 🙈

git-state

all ok

Please look at the run or in the Checks tab.

github-actions · 2023-07-10T13:00:24Z

💥 CI tests failed 🙈

git-state

all ok

Please look at the run or in the Checks tab.

Tyk-ITS · 2023-07-10T13:05:56Z

API tests result: success ✅
Branch used: refs/pull/5294/merge
Commit: 0b50bb7
Triggered by: pull_request (@mativm02)
Execution page

Tyk-ITS · 2023-07-10T13:07:47Z

API tests result: success ✅
Branch used: refs/pull/5294/merge
Commit: 6027a77
Triggered by: pull_request (@mativm02)
Execution page

Tyk-ITS · 2023-07-10T13:15:10Z

API tests result: success ✅
Branch used: refs/pull/5294/merge
Commit: b828b57
Triggered by: pull_request (@mativm02)
Execution page

github-actions · 2023-07-10T13:51:49Z

💥 CI tests failed 🙈

git-state

all ok

Please look at the run or in the Checks tab.

github-actions · 2023-07-11T11:40:52Z

💥 CI tests failed 🙈

git-state

diff --git a/go.sum b/go.sum
index bc9b573..1f8e02e 100644
--- a/go.sum
+++ b/go.sum
@@ -99,8 +99,6 @@ github.com/TykTechnologies/murmur3 v0.0.0-20230310161213-aad17efd5632 h1:T5NWziF
 github.com/TykTechnologies/murmur3 v0.0.0-20230310161213-aad17efd5632/go.mod h1:UsPYgOFBpNzDXLEti7MKOwHLpVSqdzuNGkVFPspQmnQ=
 github.com/TykTechnologies/openid2go v0.1.2 h1:WXctksOahA/epTVVvbn9iNUuMXKRr0ksrF4dY9KW8o8=
 github.com/TykTechnologies/openid2go v0.1.2/go.mod h1:gYfkqeWa+lY3Xz/Z2xYtIzmYXynlgKZaBIbPCqdcdMA=
-github.com/TykTechnologies/opentelemetry v0.0.5 h1:oH21VUJJ4wW9RlyaXIIL2Av139qRKAkUahaozSkZURg=
-github.com/TykTechnologies/opentelemetry v0.0.5/go.mod h1:bltWc1OyxnIyiado+XTl85g4EbMiCUjH52W12rbW194=
 github.com/TykTechnologies/opentelemetry v0.0.6-0.20230710132847-4d8b5d5a0515 h1:gz5nvYPmSWS186uQPrLHFhXDhjxDoyUoz3dc1sqdad0=
 github.com/TykTechnologies/opentelemetry v0.0.6-0.20230710132847-4d8b5d5a0515/go.mod h1:bltWc1OyxnIyiado+XTl85g4EbMiCUjH52W12rbW194=
 github.com/TykTechnologies/storage v1.0.5 h1:lfMljPueySAW7Mpc70g1/qC5n2LKNcKgQs+Xw30apP8=

Please look at the run or in the Checks tab.

github-actions · 2023-07-11T11:46:37Z

💥 CI tests failed 🙈

git-state

diff --git a/go.sum b/go.sum
index bc9b573..1f8e02e 100644
--- a/go.sum
+++ b/go.sum
@@ -99,8 +99,6 @@ github.com/TykTechnologies/murmur3 v0.0.0-20230310161213-aad17efd5632 h1:T5NWziF
 github.com/TykTechnologies/murmur3 v0.0.0-20230310161213-aad17efd5632/go.mod h1:UsPYgOFBpNzDXLEti7MKOwHLpVSqdzuNGkVFPspQmnQ=
 github.com/TykTechnologies/openid2go v0.1.2 h1:WXctksOahA/epTVVvbn9iNUuMXKRr0ksrF4dY9KW8o8=
 github.com/TykTechnologies/openid2go v0.1.2/go.mod h1:gYfkqeWa+lY3Xz/Z2xYtIzmYXynlgKZaBIbPCqdcdMA=
-github.com/TykTechnologies/opentelemetry v0.0.5 h1:oH21VUJJ4wW9RlyaXIIL2Av139qRKAkUahaozSkZURg=
-github.com/TykTechnologies/opentelemetry v0.0.5/go.mod h1:bltWc1OyxnIyiado+XTl85g4EbMiCUjH52W12rbW194=
 github.com/TykTechnologies/opentelemetry v0.0.6-0.20230710132847-4d8b5d5a0515 h1:gz5nvYPmSWS186uQPrLHFhXDhjxDoyUoz3dc1sqdad0=
 github.com/TykTechnologies/opentelemetry v0.0.6-0.20230710132847-4d8b5d5a0515/go.mod h1:bltWc1OyxnIyiado+XTl85g4EbMiCUjH52W12rbW194=
 github.com/TykTechnologies/storage v1.0.5 h1:lfMljPueySAW7Mpc70g1/qC5n2LKNcKgQs+Xw30apP8=

Please look at the run or in the Checks tab.

github-actions · 2023-07-11T11:58:12Z

💥 CI tests failed 🙈

git-state

all ok

Please look at the run or in the Checks tab.

github-actions · 2023-07-11T11:58:29Z

💥 CI tests failed 🙈

git-state

all ok

Please look at the run or in the Checks tab.

Tyk-ITS · 2023-07-11T12:12:21Z

API tests result: success ✅
Branch used: refs/pull/5294/merge
Commit: 6bd64b7
Triggered by: pull_request (@mativm02)
Execution page

Tyk-ITS · 2023-07-21T12:43:03Z

API tests result: success ✅
Branch used: refs/pull/5294/merge
Commit: 1ee9a1f
Triggered by: pull_request (@mativm02)
Execution page

Tyk-ITS · 2023-07-21T12:46:34Z

API tests result: success ✅
Branch used: refs/pull/5294/merge
Commit: 3ff2a8b
Triggered by: pull_request (@mativm02)
Execution page

mativm02 · 2023-07-24T12:53:05Z

Taskfile.yml

@@ -13,8 +13,7 @@ tasks:
    sources:
      - go.mod
      - go.sum
-      - './**/*.go'
-
+      - "./**/*.go"


change made by the linter

mativm02 · 2023-07-24T12:53:53Z

Taskfile.yml

+        set -e
+        task test:opentelemetry:setup
+        trap 'task test:opentelemetry:teardown' EXIT
+        task test:opentelemetry:test


I'm ensuring teardown is executed no matter if the test failed or not

Tyk-ITS · 2023-07-24T13:12:37Z

API tests result: success ✅
Branch used: refs/pull/5294/merge
Commit: c3284a5
Triggered by: pull_request (@mativm02)
Execution page

tbuchaillot

Please add a test case here with detailed spans https://github.com/TykTechnologies/tyk/blob/master/gateway/api_loader_test.go#L434

Also, please check with OAS API defs

tbuchaillot · 2023-07-24T13:43:59Z

gateway/middleware.go

+				defer span.End()
+				setContext(r, ctx)
+			} else {
+				span = tyktrace.SpanFromContext(r.Context())


Move this function to otel internal pkg, so we don't have to import /opentelemetry/trace here

tbuchaillot · 2023-07-24T13:44:30Z

apidef/schema.go

@@ -724,7 +724,10 @@ const Schema = `{
        },
 		"is_oas": {
 			"type": "boolean"
-		}
+		},
+        "detailed_tracing": {


Is this linted?

sredxny

✅ lgtm! only fix what Tom suggested

Tyk-ITS · 2023-07-24T14:47:45Z

API tests result: success ✅
Branch used: refs/pull/5294/merge
Commit: 6b84656
Triggered by: pull_request (@mativm02)
Execution page

github-actions · 2023-07-25T14:22:19Z

💥 CI tests failed 🙈

git-state

diff --git a/apidef/oas/schema/x-tyk-gateway.md b/apidef/oas/schema/x-tyk-gateway.md
index 9228f5e..89aac77 100644
--- a/apidef/oas/schema/x-tyk-gateway.md
+++ b/apidef/oas/schema/x-tyk-gateway.md
@@ -306,6 +306,9 @@ CustomDomain is the domain to bind this API to.
 
 Tyk classic API definition: `domain`.
 
+**Field: `detailedTracing` ([DetailedTracing](#detailedtracing))**
+DetailedTracing enables detailed tracing for this API.
+
 
 ### **ListenPath**
 
@@ -617,6 +620,12 @@ Enabled allow/disallow the usage of the domain.
 Name is the name of the domain.
 
 
+### **DetailedTracing**
+
+**Field: `enabled` (`boolean`)**
+Enabled enables detailed tracing.
+
+
 ### **Middleware**
 
 **Field: `global` ([Global](#global))**

Please look at the run or in the Checks tab.

github-actions · 2023-07-25T14:24:38Z

💥 CI tests failed 🙈

git-state

diff --git a/apidef/oas/schema/x-tyk-gateway.md b/apidef/oas/schema/x-tyk-gateway.md
index 9228f5e..89aac77 100644
--- a/apidef/oas/schema/x-tyk-gateway.md
+++ b/apidef/oas/schema/x-tyk-gateway.md
@@ -306,6 +306,9 @@ CustomDomain is the domain to bind this API to.
 
 Tyk classic API definition: `domain`.
 
+**Field: `detailedTracing` ([DetailedTracing](#detailedtracing))**
+DetailedTracing enables detailed tracing for this API.
+
 
 ### **ListenPath**
 
@@ -617,6 +620,12 @@ Enabled allow/disallow the usage of the domain.
 Name is the name of the domain.
 
 
+### **DetailedTracing**
+
+**Field: `enabled` (`boolean`)**
+Enabled enables detailed tracing.
+
+
 ### **Middleware**
 
 **Field: `global` ([Global](#global))**

Please look at the run or in the Checks tab.

Tyk-ITS · 2023-07-25T14:39:01Z

API tests result: success ✅
Branch used: refs/pull/5294/merge
Commit: 7f36f49
Triggered by: pull_request (@mativm02)
Execution page

Tyk-ITS · 2023-07-25T14:43:50Z

API tests result: success ✅
Branch used: refs/pull/5294/merge
Commit: 6de24ee
Triggered by: pull_request (@mativm02)
Execution page

Tyk-ITS · 2023-07-25T14:47:58Z

API tests result: success ✅
Branch used: refs/pull/5294/merge
Commit: 6f5619b
Triggered by: pull_request (@mativm02)
Execution page

Tyk-ITS · 2023-07-25T15:52:15Z

API tests result: success ✅
Branch used: refs/pull/5294/merge
Commit: b17b01a
Triggered by: pull_request (@mativm02)
Execution page

Tyk-ITS · 2023-07-31T14:30:05Z

API tests result: success ✅
Branch used: refs/pull/5294/merge
Commit: b3e4a60
Triggered by: pull_request (@mativm02)
Execution page

sonarcloud · 2023-07-31T14:35:47Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
1 Code Smell

88.9% Coverage
0.0% Duplication

Tyk-ITS · 2023-07-31T14:40:08Z

API tests result: success ✅
Branch used: refs/pull/5294/merge
Commit: 6b84656
Triggered by: pull_request (@mativm02)
Execution page

Tyk-ITS · 2023-07-31T14:40:51Z

API tests result: success ✅
Branch used: refs/pull/5294/merge
Commit: 8c83546
Triggered by: pull_request (@mativm02)
Execution page

Detailed Tracing

1d86a23

reverting transport changes

10c54c7

Merge branch 'master' into TT-7902

45f08e1

mativm02 marked this pull request as ready for review July 7, 2023 14:56

mativm02 added 3 commits July 10, 2023 09:44

removing omitempty from json tag

cc5fb1d

Merge branch 'TT-7902' of github.com:TykTechnologies/tyk into TT-7902

6027a77

Merge branch 'master' into TT-7902

0b50bb7

Merge branch 'master' into TT-7902

b828b57

mativm02 added 2 commits July 11, 2023 08:35

saving errors on non-detailed traces as well

f1df8e1

Merge branch 'TT-7902' of github.com:TykTechnologies/tyk into TT-7902

eaa1139

mativm02 added 2 commits July 11, 2023 08:51

linting

6bd64b7

Merge branch 'master' into TT-7902

f0d6d7c

mativm02 added 2 commits July 11, 2023 09:02

fixing test

8bd46b4

Merge branch 'TT-7902' of github.com:TykTechnologies/tyk into TT-7902

b24ad00

mativm02 added 4 commits July 21, 2023 13:09

Merge branch 'master' of github.com:TykTechnologies/tyk into TT-7902

a175801

Merge branch 'master' of github.com:TykTechnologies/tyk into TT-7902

3120cdc

adding e2e tests

84ef4de

Merge branch 'TT-7902' of github.com:TykTechnologies/tyk into TT-7902

c3284a5

mativm02 commented Jul 24, 2023

View reviewed changes

tbuchaillot requested changes Jul 24, 2023

View reviewed changes

moving imports to internal package

6b84656

sredxny approved these changes Jul 24, 2023

View reviewed changes

alephnull removed the request for review from a team July 26, 2023 11:28

mativm02 force-pushed the TT-7902 branch 2 times, most recently from b3e4a60 to 6b84656 Compare July 31, 2023 14:18

Merge branch 'master' into TT-7902

8c83546

tbuchaillot approved these changes Jul 31, 2023

View reviewed changes

mativm02 merged commit 7fa737d into master Jul 31, 2023
18 of 19 checks passed

mativm02 deleted the TT-7902 branch July 31, 2023 17:15

[TT-7902] Detailed Tracing #5294

[TT-7902] Detailed Tracing #5294

Conversation

mativm02 commented Jul 7, 2023 • edited Loading

Description

Related Issue

Motivation and Context

How This Has Been Tested

Screenshots (if appropriate)

Types of changes

Checklist

Tyk-ITS commented Jul 7, 2023

Tyk-ITS commented Jul 7, 2023

github-actions bot commented Jul 7, 2023

💥 CI tests failed 🙈

git-state

Tyk-ITS commented Jul 7, 2023

github-actions bot commented Jul 10, 2023

💥 CI tests failed 🙈

git-state

github-actions bot commented Jul 10, 2023

💥 CI tests failed 🙈

git-state

github-actions bot commented Jul 10, 2023

💥 CI tests failed 🙈

git-state

Tyk-ITS commented Jul 10, 2023

Tyk-ITS commented Jul 10, 2023

Tyk-ITS commented Jul 10, 2023

github-actions bot commented Jul 10, 2023

💥 CI tests failed 🙈

git-state

github-actions bot commented Jul 11, 2023

💥 CI tests failed 🙈

git-state

github-actions bot commented Jul 11, 2023

💥 CI tests failed 🙈

git-state

github-actions bot commented Jul 11, 2023

💥 CI tests failed 🙈

git-state

github-actions bot commented Jul 11, 2023

💥 CI tests failed 🙈

git-state

Tyk-ITS commented Jul 11, 2023

Tyk-ITS commented Jul 21, 2023

Tyk-ITS commented Jul 21, 2023

mativm02 Jul 24, 2023

Choose a reason for hiding this comment

mativm02 Jul 24, 2023

Choose a reason for hiding this comment

Tyk-ITS commented Jul 24, 2023

tbuchaillot left a comment • edited Loading

Choose a reason for hiding this comment

tbuchaillot Jul 24, 2023

Choose a reason for hiding this comment

tbuchaillot Jul 24, 2023

Choose a reason for hiding this comment

mativm02 Jul 25, 2023

Choose a reason for hiding this comment

sredxny left a comment

Choose a reason for hiding this comment

Tyk-ITS commented Jul 24, 2023

github-actions bot commented Jul 25, 2023

💥 CI tests failed 🙈

git-state

github-actions bot commented Jul 25, 2023

💥 CI tests failed 🙈

git-state

Tyk-ITS commented Jul 25, 2023

Tyk-ITS commented Jul 25, 2023

Tyk-ITS commented Jul 25, 2023

Tyk-ITS commented Jul 25, 2023

Tyk-ITS commented Jul 31, 2023

sonarcloud bot commented Jul 31, 2023

Tyk-ITS commented Jul 31, 2023

Tyk-ITS commented Jul 31, 2023

mativm02 commented Jul 7, 2023 •

edited

Loading

tbuchaillot left a comment •

edited

Loading