feat: Implemented ability to dynamically reload TLS certificates

[cdoucy]

Fixes #856

[cdoucy]

Hey @jeevatkm !

As discussed, here is the PR.

If you are happy with the changes, I can update doc.

[jeevatkm]

@cdoucy Thanks for creating a PR. Please take a look at my feedback.

• Make use of https://pkg.go.dev/time#Ticker to execute the function to reload based on a timer
• Do not use OnBeforeRequest, instead keep the loading directly into the client instance TLS config
• Do not combine RootCA and ClientCA certs pool; keep them separate by nature
• 1 minute seems very frequent; certificate refresh only happens occasionally in a practical scenario. For now, let's keep 24 hours; then user can decide on it
• Keep pem refresher as part of client struct. Once it becomes part of the client, then we do not need lastChecked, log, debug, and certPool as part of the watcher

[cdoucy]

Hi @jeevatkm thanks for the feedback!

Make use of https://pkg.go.dev/time#Ticker to execute the function to reload based on a timer

yes, but that means that ticker.Stop() needs to be called at some point.
For this, SetRootCertificateWatcher may accept a context.Context, and ticker.Stop() will be called when the context is called.
What do you think?

keep the loading directly into the client instance TLS config

Note sure to understand what you mean here.
To avoid using OnBeforeRequest, I can implement an http.RoundTripper that would get the refreshed Certificate and pass it to the TLSClientConfig of the underlying http.Transporter.
What do you think of this approach?

P.S. : I wish I could leverage tls.Config.GetCertificate, but it only works on the server side.

[jeevatkm]

@cdoucy I'm not 100% sure, do we really need a context for this use case?

I may try to explain an approach based on what I learned from your PR and currently what is coming to my mind. This is not perfect; I am just dumping my thoughts.

type CertWatcherOptions struct {
	// Default interval is 24 hours
	PoolInterval time.Duration
}

func (c *Client) SetRootCertificateWatcher(certPath string, options *CertWatcherOptions) *Client {
	c.SetRootCertificate(certPath)
	c.initRootCertWatcher(options)
 	return c
}

func (c *Client) SetClientRootCertificateWatcher(certPath string, options *CertWatcherOptions) *Client {
	c.SetClientRootCertificate(certPath)
	c.initClientRootCertWatcher(options)
 	return c
}

func (c *Client) initRootCertWatcher() {
	c.rootCAWatcher = &pemWatcher{
		// ...
	}

	// ...
}

func (c *Client) initClientRootCertWatcher() {
	c.clientRootCAWatcher = &pemWatcher{
		// ...
	}

	// ...	
}

So, that user could call SetRootCertificateWatcher method to set N number cert files, and those get added to the watch list
...
Let's assume there is a method c.refreshRootCerts() and c.refreshClientRootCerts() gets triggered by time.Ticker and reloads the certs into c.tlsConfig().RootCAs and c.tlsConfig().ClientCAs, respectively.

We can add a method to the Client Stop cert watcher ticker and add documentation for users to call that method from their shutdown hook scenarios.

What do you think?

[jeevatkm]

@cdoucy, I thought I would check with you about the last comment.

[cdoucy]

Hello @jeevatkm,

Thanks for your feedback! Sorry for the delayed response.
Your approach makes sense to me.
If a Stop() method can be introduced in the client, then we don't need a context.

I'll work on applying your suggestion.

[jeevatkm]

@cdoucy, Can you please implement it against the main branch for Resty v3? I plan to introduce the method Close at the client level to tackle all the channels, tickers, etc. Clean-up could be done on that.

Then the user would do something like this v3 onwards -

client := resty.New()
defer client.Close()
...

[cdoucy]

Hi @jeevatkm , sure!
I'll work on this over the weekend.

[cdoucy]

Hi @jeevatkm !

Hope you are doing well.

I've made the following change to the PR:

• Rebased the branch off main
• Moved reloader code to client struct
• Refactored the code to use a time.Ticker
• Set the default pooling interval to 24 hours

[jeevatkm]

@cdoucy Thanks for updating PR. The implementation looks good.
I have added one comment and one thought; please have a look.

[jeevatkm]

@cdoucy, I wonder if we need an additional channel to stop the ticker. I will think about it.

[cdoucy]

The other approach I considered is to maintain a slice that would keep track of all time.Ticker instances, and the Close() method would iterate over this slice and stop all the tickers.

I judged that the channel approach is easier to implement and more in line with golang idiomatics.

Do you see any other approaches?

[jeevatkm]

I didn't get a chance to look at it deeper. We can keep the current approach. If needed, I can improve it afterward.

[jeevatkm]

@cdoucy Thanks for updating the PR.

[jeevatkm]

@cdoucy It seems there is a test case failure. Can you please take care of it?

[cdoucy]

@cdoucy It seems there is a test case failure. Can you please take care of it?

Hi @jeevatkm I may have fixed it with 0959aa0

[codecov]

Codecov Report

Attention: Patch coverage is 71.66667% with 17 lines in your changes missing coverage. Please review.

Project coverage is 97.50%. Comparing base (c6cf259) to head (a79d65f).
Report is 9 commits behind head on main.

Files with missing lines	Patch %	Lines
client.go	71.18%	14 Missing and 3 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #866      +/-   ##
==========================================
- Coverage   98.15%   97.50%   -0.65%     
==========================================
  Files          14       16       +2     
  Lines        2329     3171     +842     
==========================================
+ Hits         2286     3092     +806     
- Misses         23       56      +33     
- Partials       20       23       +3     

Flag	Coverage Δ
unittests	97.50% <71.66%> (-0.65%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[jeevatkm]

@cdoucy Thanks for fixing the test cases. Can you please look at the missed lines in the test coverage?
https://github.com/go-resty/resty/pull/866/checks?check_run_id=31794960778