[TT-13766] Bump newrelic dependency (#6809)

### **User description** <details open> <summary><a href="https://tyktech.atlassian.net/browse/TT-13766" title="TT-13766" target="_blank">TT-13766</a></summary> <br /> <table> <tr> <th>Summary</th> <td>Bump newrelic dependency</td> </tr> <tr> <th>Type</th> <td> <img alt="Story" src="https://tyktech.atlassian.net/rest/api/2/universal_avatar/view/type/issuetype/avatar/10315?size=medium" /> Story </td> </tr> <tr> <th>Status</th> <td>In Code Review</td> </tr> <tr> <th>Points</th> <td>N/A</td> </tr> <tr> <th>Labels</th> <td><a href="https://tyktech.atlassian.net/issues?jql=project%20%3D%20TT%20AND%20labels%20%3D%20SESAP%20ORDER%20BY%20created%20DESC" title="SESAP">SESAP</a></td> </tr> </table> </details>  --- This PR bumps the dependency with supposedly minimal changes, following the extensive upgrading guide. Needs an e2e test with newrelic (see ticket). https://tyktech.atlassian.net/browse/TT-13766 ___ ### **PR Type** Enhancement, Dependencies ___ ### **Description** - Migrated New Relic integration to use the updated v3 library. - Introduced a new `internal/service/newrelic` package for centralized New Relic functionality. - Refactored middleware, server, and proxy muxer to use the new context-based transaction handling. - Removed the old `gateway/newrelic.go` file and its outdated implementation. - Updated `go.mod` and `go.sum` to include the new New Relic v3 library and `nrgorilla` integration. ___ ### **Changes walkthrough** 📝 <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><table> <tr> <td> <details> <summary><strong>middleware.go</strong><dd><code>Refactor middleware to use updated New Relic context-based API</code></dd></summary> <hr> gateway/middleware.go <li>Replaced direct usage of <code>newrelic.Transaction</code> with a new context-based <br>approach using <code>newrelic.Context</code>.<br> <li> Updated middleware logic to use the new <code>StartSegment</code> method from the <br>updated New Relic library.<br> <li> Adjusted imports to use the new internal <code>service/newrelic</code> package.<br> </details> </td> <td><a href="https://github.com/TykTechnologies/tyk/pull/6809/files#diff-703054910891a4db633eca0f42ed779d6b4fa75cd9b3aa4c503e681364201c1b">+3/-3</a>      </td> </tr> <tr> <td> <details> <summary><strong>newrelic.go</strong><dd><code>Remove old New Relic setup and instrumentation logic</code>          </dd></summary> <hr> gateway/newrelic.go <li>Removed the old implementation of New Relic setup and instrumentation.<br> <li> Deprecated the file as its functionality has been moved to the new <br><code>internal/service/newrelic</code> package.<br> </details> </td> <td><a href="https://github.com/TykTechnologies/tyk/pull/6809/files#diff-5e962a38f6108a1954500d7e078fdafe5d53f22c6ba058af7afa4dee4b99a1e2">+0/-100</a>  </td> </tr> <tr> <td> <details> <summary><strong>proxy_muxer.go</strong><dd><code>Update proxy muxer to use new New Relic context handling</code>  </dd></summary> <hr> gateway/proxy_muxer.go <li>Updated transaction handling to use the new <code>newrelic.Context</code> for <br>setting and retrieving transactions.<br> <li> Adjusted imports to use the new <code>service/newrelic</code> package.<br> </details> </td> <td><a href="https://github.com/TykTechnologies/tyk/pull/6809/files#diff-89fb6731880400cb95ba8860c935a308de5f55aaa41aa2c76abf3ee4773d7a87">+7/-2</a>      </td> </tr> <tr> <td> <details> <summary><strong>server.go</strong><dd><code>Refactor server New Relic setup to use updated library</code>      </dd></summary> <hr> gateway/server.go <li>Refactored <code>SetupNewRelic</code> to use the updated New Relic library and <br>configuration options.<br> <li> Updated the global <code>NewRelicApplication</code> variable to use the new <br><code>*newrelic.Application</code> type.<br> <li> Integrated the new <code>service/newrelic</code> package for New Relic <br>functionality.<br> </details> </td> <td><a href="https://github.com/TykTechnologies/tyk/pull/6809/files#diff-4652d1bf175a0be8f5e61ef7177c9666f23e077d8626b73ac9d13358fa8b525b">+31/-2</a>    </td> </tr> <tr> <td> <details> <summary><strong>newrelic.go</strong><dd><code>Add new service package for New Relic integration</code>                </dd></summary> <hr> internal/service/newrelic/newrelic.go <li>Introduced a new package for handling New Relic integration.<br> <li> Added context-based transaction management and logging utilities.<br> <li> Implemented a new sink for emitting custom events and metrics to New <br>Relic.<br> </details> </td> <td><a href="https://github.com/TykTechnologies/tyk/pull/6809/files#diff-7c791ca90be92a2cab3f0b458e287d89da843e75aa60147f1cad613a5debb56e">+99/-0</a>    </td> </tr> </table></td></tr><tr><td><strong>Dependencies</strong></td><td><table> <tr> <td> <details> <summary><strong>go.mod</strong><dd><code>Update dependencies for New Relic v3 and integrations</code>        </dd></summary> <hr> go.mod <li>Updated New Relic dependency to version 3.<br> <li> Added <code>nrgorilla</code> integration for New Relic.<br> </details> </td> <td><a href="https://github.com/TykTechnologies/tyk/pull/6809/files#diff-33ef32bf6c23acb95f5902d7097b7a1d5128ca061167ec0716715b0b9eeaa5f6">+2/-1</a>      </td> </tr> <tr> <td> <details> <summary><strong>go.sum</strong><dd><code>Update dependency checksums for New Relic v3</code>                          </dd></summary> <hr> go.sum <li>Updated checksums for the new New Relic v3 and <code>nrgorilla</code> dependencies.<br> <br> </details> </td> <td><a href="https://github.com/TykTechnologies/tyk/pull/6809/files#diff-3295df7234525439d778f1b282d146a4f1ff6b415248aaac074e8042d9f42d63">+4/-2</a>      </td> </tr> </table></td></tr></tr></tbody></table> ___ > 💡 **PR-Agent usage**: Comment `/help "your question"` on any pull request to receive relevant information --------- Co-authored-by: Tit Petric <[email protected]>
TykTechnologies · Jan 3, 2025 · acb2e19 · acb2e19
1 parent f65d41a
commit acb2e19
Show file tree

Hide file tree

Showing 11 changed files with 166 additions and 118 deletions.
diff --git a/gateway/api_loader.go b/gateway/api_loader.go
@@ -29,6 +29,7 @@ import (
 	"github.com/TykTechnologies/tyk/internal/httpctx"
 	"github.com/TykTechnologies/tyk/internal/httputil"
 	"github.com/TykTechnologies/tyk/internal/otel"
+	"github.com/TykTechnologies/tyk/internal/service/newrelic"
 )
 
 const (
@@ -769,6 +770,8 @@ func (gw *Gateway) loadHTTPService(spec *APISpec, apisByListen map[string]int, g
 	router := muxer.router(port, spec.Protocol, gwConfig)
 	if router == nil {
 		router = mux.NewRouter()
+		newrelic.Mount(router, gw.NewRelicApplication)
+
 		muxer.setRouter(port, spec.Protocol, router, gwConfig)
 	}
 

diff --git a/gateway/middleware.go b/gateway/middleware.go
@@ -15,7 +15,6 @@ import (
 
 	"github.com/gocraft/health"
 	"github.com/justinas/alice"
-	newrelic "github.com/newrelic/go-agent"
 	"github.com/paulbellamy/ratecounter"
 	"github.com/sirupsen/logrus"
 	"golang.org/x/sync/singleflight"
@@ -27,6 +26,7 @@ import (
 	"github.com/TykTechnologies/tyk/internal/middleware"
 	"github.com/TykTechnologies/tyk/internal/otel"
 	"github.com/TykTechnologies/tyk/internal/policy"
+	"github.com/TykTechnologies/tyk/internal/service/newrelic"
 	"github.com/TykTechnologies/tyk/request"
 	"github.com/TykTechnologies/tyk/rpc"
 	"github.com/TykTechnologies/tyk/storage"
@@ -137,10 +137,8 @@ func (gw *Gateway) createMiddleware(actualMW TykMiddleware) func(http.Handler) h
 		return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 			logger := mw.Base().SetRequestLogger(r)
 
-			if gw.GetConfig().NewRelic.AppName != "" {
-				if txn, ok := w.(newrelic.Transaction); ok {
-					defer newrelic.StartSegment(txn, mw.Name()).End()
-				}
+			if txn := newrelic.FromContext(r.Context()); txn != nil {
+				defer txn.StartSegment(mw.Name()).End()
 			}
 
 			job := instrument.NewJob("MiddlewareCall")

diff --git a/gateway/newrelic.go b/gateway/newrelic.go
diff --git a/gateway/proxy_muxer.go b/gateway/proxy_muxer.go
@@ -95,12 +95,6 @@ func (h *handleWrapper) ServeHTTP(w http.ResponseWriter, r *http.Request) {
 		return
 	}
 
-	if NewRelicApplication != nil {
-		txn := NewRelicApplication.StartTransaction(r.URL.Path, w, r)
-		defer txn.End()
-		h.router.ServeHTTP(txn, r)
-		return
-	}
 	h.router.ServeHTTP(w, r)
 }
 

diff --git a/gateway/server.go b/gateway/server.go
@@ -36,7 +36,6 @@ import (
 	grayloghook "github.com/gemnasium/logrus-graylog-hook"
 	"github.com/gorilla/mux"
 	"github.com/lonelycode/osin"
-	newrelic "github.com/newrelic/go-agent"
 	"github.com/sirupsen/logrus"
 	logrussyslog "github.com/sirupsen/logrus/hooks/syslog"
 
@@ -67,6 +66,7 @@ import (
 	"github.com/TykTechnologies/tyk/internal/cache"
 	"github.com/TykTechnologies/tyk/internal/model"
 	"github.com/TykTechnologies/tyk/internal/netutil"
+	"github.com/TykTechnologies/tyk/internal/service/newrelic"
 )
 
 var (
@@ -77,8 +77,7 @@ var (
 	pubSubLog = log.WithField("prefix", "pub-sub")
 	rawLog    = logger.GetRaw()
 
-	memProfFile         *os.File
-	NewRelicApplication newrelic.Application
+	memProfFile *os.File
 
 	// confPaths is the series of paths to try to use as config files. The
 	// first one to exist will be used. If none exists, a default config
@@ -125,6 +124,7 @@ type Gateway struct {
 	HostCheckTicker      chan struct{}
 	HostCheckerClient    *http.Client
 	TracerProvider       otel.TracerProvider
+	NewRelicApplication  *newrelic.Application
 
 	keyGen DefaultKeyGenerator
 
@@ -256,6 +256,33 @@ func NewGateway(config config.Config, ctx context.Context) *Gateway {
 	return gw
 }
 
+// SetupNewRelic creates new newrelic.Application instance.
+func (gw *Gateway) SetupNewRelic() (app *newrelic.Application) {
+	var (
+		err      error
+		gwConfig = gw.GetConfig()
+	)
+
+	log := log.WithFields(logrus.Fields{"prefix": "newrelic"})
+
+	cfg := []newrelic.ConfigOption{
+		newrelic.ConfigAppName(gwConfig.NewRelic.AppName),
+		newrelic.ConfigLicense(gwConfig.NewRelic.LicenseKey),
+		newrelic.ConfigEnabled(gwConfig.NewRelic.AppName != ""),
+		newrelic.ConfigDistributedTracerEnabled(gwConfig.NewRelic.EnableDistributedTracing),
+		newrelic.ConfigLogger(newrelic.NewLogger(log)),
+	}
+
+	if app, err = newrelic.NewApplication(cfg...); err != nil {
+		log.Warn("Error initializing NewRelic, skipping... ", err)
+		return
+	}
+
+	instrument.AddSink(newrelic.NewSink(app))
+
+	return
+}
+
 func (gw *Gateway) UnmarshalJSON(data []byte) error {
 	return nil
 }
@@ -440,7 +467,7 @@ func (gw *Gateway) setupGlobals() {
 	}
 
 	if gw.GetConfig().NewRelic.AppName != "" {
-		NewRelicApplication = gw.SetupNewRelic()
+		gw.NewRelicApplication = gw.SetupNewRelic()
 	}
 
 	gw.readGraphqlPlaygroundTemplate()

diff --git a/gateway/testutil.go b/gateway/testutil.go
@@ -1161,6 +1161,10 @@ func (s *Test) newGateway(genConf func(globalConf *config.Config)) *Gateway {
 	gwConfig.BundleBaseURL = testHttpBundles
 	gwConfig.MiddlewarePath = testMiddlewarePath
 
+	if err := config.FillEnv(&gwConfig); err != nil {
+		log.WithError(err).Error("error filling test config from env")
+	}
+
 	// force ipv4 for now, to work around the docker bug affecting
 	// Go 1.8 and earlier
 	gwConfig.ListenAddress = "127.0.0.1"
@@ -1309,6 +1313,7 @@ func (s *Test) Close() {
 	s.Gw.Analytics.Stop()
 	s.Gw.ReloadTestCase.StopTicker()
 	s.Gw.GlobalHostChecker.StopPoller()
+	s.Gw.NewRelicApplication.Shutdown(5 * time.Second)
 
 	err = s.RemoveApis()
 	if err != nil {

diff --git a/go.mod b/go.mod
@@ -92,7 +92,8 @@ require (
 	github.com/goccy/go-json v0.10.4
 	github.com/google/go-cmp v0.6.0
 	github.com/nats-io/nats.go v1.38.0
-	github.com/newrelic/go-agent v2.13.0+incompatible
+	github.com/newrelic/go-agent/v3 v3.35.1
+	github.com/newrelic/go-agent/v3/integrations/nrgorilla v1.2.2
 	github.com/testcontainers/testcontainers-go v0.34.0
 	github.com/testcontainers/testcontainers-go/modules/kafka v0.33.0
 	github.com/testcontainers/testcontainers-go/modules/nats v0.33.0

diff --git a/go.sum b/go.sum
@@ -604,8 +604,10 @@ github.com/nats-io/nuid v1.0.1 h1:5iA8DT8V7q8WK2EScv2padNa/rTESc1KdnPw4TC2paw=
 github.com/nats-io/nuid v1.0.1/go.mod h1:19wcPz3Ph3q0Jbyiqsd0kePYG7A95tJPxeL+1OSON2c=
 github.com/nats-io/stan.go v0.10.4 h1:19GS/eD1SeQJaVkeM9EkvEYattnvnWrZ3wkSWSw4uXw=
 github.com/nats-io/stan.go v0.10.4/go.mod h1:3XJXH8GagrGqajoO/9+HgPyKV5MWsv7S5ccdda+pc6k=
-github.com/newrelic/go-agent v2.13.0+incompatible h1:Dl6m75MHAzfB0kicv9GiLxzQatRjTLUAdrnYyoT8s4M=
-github.com/newrelic/go-agent v2.13.0+incompatible/go.mod h1:a8Fv1b/fYhFSReoTU6HDkTYIMZeSVNffmoS726Y0LzQ=
+github.com/newrelic/go-agent/v3 v3.35.1 h1:N43qBNDILmnwLDCSfnE1yy6adyoVEU95nAOtdUgG4vA=
+github.com/newrelic/go-agent/v3 v3.35.1/go.mod h1:GNTda53CohAhkgsc7/gqSsJhDZjj8vaky5u+vKz7wqM=
+github.com/newrelic/go-agent/v3/integrations/nrgorilla v1.2.2 h1:YaFf6tmxSKNVgS9ZHx6O8HSpckWiyNSBZQKwaXfG1fQ=
+github.com/newrelic/go-agent/v3/integrations/nrgorilla v1.2.2/go.mod h1:NlYWXdP4WVAg8v7ZM0FRWulv0OtssOS3l4R6pYlWGf0=
 github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e/go.mod h1:zD1mROLANZcx1PVRCS0qkT7pwLkGfwJo4zjcN/Tysno=
 github.com/nsf/jsondiff v0.0.0-20230430225905-43f6cf3098c1 h1:dOYG7LS/WK00RWZc8XGgcUTlTxpp3mKhdR2Q9z9HbXM=
 github.com/nsf/jsondiff v0.0.0-20230430225905-43f6cf3098c1/go.mod h1:mpRZBD8SJ55OIICQ3iWH0Yz3cjzA61JdqMLoWXeB2+8=

diff --git a/internal/service/newrelic/logger.go b/internal/service/newrelic/logger.go
@@ -0,0 +1,33 @@
+package newrelic
+
+import (
+	"github.com/newrelic/go-agent/v3/newrelic"
+
+	"github.com/sirupsen/logrus"
+)
+
+type Logger struct {
+	*logrus.Entry
+}
+
+var _ newrelic.Logger = &Logger{}
+
+func NewLogger(e *logrus.Entry) *Logger {
+	return &Logger{e}
+}
+
+func (l *Logger) Error(msg string, c map[string]interface{}) {
+	l.WithFields(c).Error(msg)
+}
+func (l *Logger) Warn(msg string, c map[string]interface{}) {
+	l.WithFields(c).Warn(msg)
+}
+func (l *Logger) Info(msg string, c map[string]interface{}) {
+	l.WithFields(c).Info(msg)
+}
+func (l *Logger) Debug(msg string, c map[string]interface{}) {
+	l.WithFields(c).Debug(msg)
+}
+func (l *Logger) DebugEnabled() bool {
+	return l.Level >= logrus.DebugLevel
+}
diff --git a/internal/service/newrelic/newrelic.go b/internal/service/newrelic/newrelic.go
@@ -0,0 +1,36 @@
+package newrelic
+
+import (
+	"github.com/gorilla/mux"
+	"github.com/newrelic/go-agent/v3/integrations/nrgorilla"
+	"github.com/newrelic/go-agent/v3/newrelic"
+)
+
+// Type aliases used from newrelic pkg.
+type (
+	Application  = newrelic.Application
+	Transaction  = newrelic.Transaction
+	ConfigOption = newrelic.ConfigOption
+)
+
+// Variable aliases used from newrelic pkg.
+var (
+	NewApplication = newrelic.NewApplication
+	FromContext    = newrelic.FromContext
+
+	ConfigLogger                   = newrelic.ConfigLogger
+	ConfigEnabled                  = newrelic.ConfigEnabled
+	ConfigAppName                  = newrelic.ConfigAppName
+	ConfigLicense                  = newrelic.ConfigLicense
+	ConfigDistributedTracerEnabled = newrelic.ConfigDistributedTracerEnabled
+)
+
+// Mount adds the nrgorilla middleware to the router. The application is added to the request context.
+// If app is nil, nothing will be done and the function will return.
+func Mount(router *mux.Router, app *Application) {
+	if app == nil {
+		return
+	}
+
+	router.Use(nrgorilla.Middleware(app))
+}
diff --git a/internal/service/newrelic/sink.go b/internal/service/newrelic/sink.go
@@ -0,0 +1,49 @@
+package newrelic
+
+import (
+	"fmt"
+	"strconv"
+
+	"github.com/newrelic/go-agent/v3/newrelic"
+
+	"github.com/gocraft/health"
+)
+
+type Sink struct {
+	relic *newrelic.Application
+	health.Sink
+}
+
+func NewSink(relic *newrelic.Application) *Sink {
+	return &Sink{
+		relic: relic,
+	}
+}
+
+func (s *Sink) EmitEvent(job string, event string, kvs map[string]string) {
+	s.relic.RecordCustomEvent(job+":"+event, makeParams(kvs))
+}
+
+func (s *Sink) EmitEventErr(job string, event string, err error, kvs map[string]string) {
+	s.relic.RecordCustomEvent(job+":"+event+":msg:"+err.Error(), makeParams(kvs))
+}
+
+func (s *Sink) EmitTiming(job string, event string, nanoseconds int64, kvs map[string]string) {
+	s.relic.RecordCustomEvent(job+":"+event+":duration_ns:"+strconv.FormatInt(nanoseconds, 10), makeParams(kvs))
+}
+
+func (s *Sink) EmitComplete(job string, status health.CompletionStatus, nanoseconds int64, kvs map[string]string) {
+	s.relic.RecordCustomEvent(job+":health:"+status.String()+":duration_ns:"+strconv.FormatInt(nanoseconds, 10), makeParams(kvs))
+}
+
+func (s *Sink) EmitGauge(job string, event string, value float64, kvs map[string]string) {
+	s.relic.RecordCustomEvent(job+":"+event+":value:"+fmt.Sprintf("%.2f", value), makeParams(kvs))
+}
+
+func makeParams(kvs map[string]string) (params map[string]interface{}) {
+	params = make(map[string]interface{}, len(kvs))
+	for k, v := range kvs {
+		params[k] = v
+	}
+	return
+}