Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue 994 malformed hostnames #1004

Merged
merged 4 commits into from
Aug 21, 2024
Merged

Issue 994 malformed hostnames #1004

merged 4 commits into from
Aug 21, 2024

Conversation

ARolek
Copy link
Member

@ARolek ARolek commented Jul 19, 2024

Though this PR has a lot of code changes, a lot of them are clean up, adding consts, and formatting. The material changes are as follows:

  • Removed internal function buildCapabilitiesURL from the server package. This function was handling too many cases and brittle.
  • Added a new type to the server package: TileURLTemplate. This type is responsible for building up all the Tile template URLs. So any URL with the /{z}/{x}/{y}.pbf uri template suffix will now use this type.
  • Added hostname parsing & validation to the config package. The hostname attribute does not support a protocol, but sometimes the protocol was configured and it would result in malformed URLs (putting "https://....." in hostname config directive results in a malformed URL #994). The validation and use of the url.URL package helps work with user supplied URLs.

Overall, it's a lot of code, but I got frustrated with the state of things and deemed a refactor necessary.

@ARolek ARolek force-pushed the issue-994-malformed-hostnames branch 2 times, most recently from 20fdec2 to 41167a9 Compare July 19, 2024 23:49
@coveralls
Copy link

coveralls commented Jul 19, 2024

Pull Request Test Coverage Report for Build 1919ce1db

Details

  • 145 of 189 (76.72%) changed or added relevant lines in 12 files are covered.
  • 1 unchanged line in 1 file lost coverage.
  • Overall coverage increased (+0.3%) to 42.816%

Changes Missing Coverage Covered Lines Changed/Added Lines %
server/handle_map_style.go 7 8 87.5%
server/handle_capabilities.go 26 28 92.86%
server/server.go 5 7 71.43%
cmd/tegola_lambda/main.go 0 3 0.0%
server/errors.go 0 3 0.0%
cmd/tegola/cmd/server.go 0 4 0.0%
internal/env/parse.go 9 13 69.23%
internal/env/types.go 10 16 62.5%
server/tile_url_template.go 69 77 89.61%
config/errors.go 0 11 0.0%
Files with Coverage Reduction New Missed Lines %
cmd/tegola_lambda/main.go 1 0.0%
Totals Coverage Status
Change from base Build a58cd8965: 0.3%
Covered Lines: 7030
Relevant Lines: 16419

💛 - Coveralls

@ARolek ARolek force-pushed the issue-994-malformed-hostnames branch from 41167a9 to 55fa52d Compare July 20, 2024 00:08
@ARolek ARolek requested a review from iwpnd July 20, 2024 00:11
@ARolek ARolek marked this pull request as ready for review July 20, 2024 00:12
@ARolek ARolek requested a review from gdey as a code owner July 20, 2024 00:12
@ARolek ARolek force-pushed the issue-994-malformed-hostnames branch from 55fa52d to e38590a Compare July 20, 2024 20:11
Copy link
Member

@gdey gdey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Thank you for all the cleanup. It is good to get some of the magic values into constants.

@dBitech
Copy link

dBitech commented Aug 8, 2024

I took some time to test this out and seems to function as advertised. Look forward to it's commit bit

Copy link
Member

@iwpnd iwpnd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good. some nice cleanup. 👍

@@ -91,7 +91,12 @@ func init() {
// set our server version
server.Version = build.Version
if conf.Webserver.HostName != "" {
server.HostName = string(conf.Webserver.HostName)
hostname, err := url.Parse(string(conf.Webserver.HostName))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is redundant. If you validate the hostname in config.validate() already it will throw early.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@iwpnd you're right it is redundant. I just went back and looked at the code to determine why I did it this way and I think I'm hitting a weird issue that could probably be solved more elegantly:

  • The config package has a ValidateAndRegisterParams routine which calls the url.Parse() method on the Hostname and throws an error if the Parse fails. It does not set the value of Webserver.Hostname, which is of type env.String.

tegola/config/config.go

Lines 337 to 345 in e38590a

if string(c.Webserver.HostName) != "" {
_, err := url.Parse(string(c.Webserver.HostName))
if err != nil {
return ErrInvalidHostName{
HostName: string(c.Webserver.HostName),
Err: err,
}
}
}

  • The server.Hostname value is now of type url.URL and since config.Webserver.Hostname is of type env.String we need to parse again.

I think I opted for the double Parse approach as this code path only gets hit during start up. A more elegant way to solve this would be to add a new type to the internal/env package called URL. This should be pretty straight forward to do so I will give it a shot.

Good call out.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@iwpnd I added a new commit addressing this comment. LMK what you think.

@@ -38,9 +39,17 @@ var serverCmd = &cobra.Command{
serverPort = string(conf.Webserver.Port)
}

if conf.Webserver.HostName != "" {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this would have been caught in config.validate() already, no?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@iwpnd I added a new commit addressing this comment. LMK what you think.

@ARolek ARolek requested review from gdey and iwpnd August 17, 2024 19:32
@ARolek
Copy link
Member Author

ARolek commented Aug 17, 2024

@gdey / @iwpnd I have added another commit which implemented a new config type: env.URL. This removes the need for the URL parsing to happen at latter stages. I have tagged you two for re-review when you have a min.

config/testdata/empty_proxy_protocol.toml Show resolved Hide resolved
internal/env/parse.go Show resolved Hide resolved
internal/env/parse_test.go Show resolved Hide resolved
internal/env/types.go Show resolved Hide resolved
@ARolek ARolek requested a review from iwpnd August 19, 2024 12:04
Copy link
Member

@gdey gdey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

* Added url.Parse() validation to the webserver.hostname.
* Use stricter formatting strings in config package.
Introduced the type TileURLTemplate for handling forming up
tile template URLs. This adds more structure and type safety when
handling the creation of tile template URLs.

closes #994
This type was added for better error handling of the
Webserver.HostName config property.
@ARolek ARolek force-pushed the issue-994-malformed-hostnames branch from 1919ce1 to 287545e Compare August 20, 2024 23:03
@ARolek ARolek merged commit 287545e into master Aug 21, 2024
18 checks passed
@ARolek ARolek deleted the issue-994-malformed-hostnames branch August 21, 2024 00:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

putting "https://....." in hostname config directive results in a malformed URL
5 participants