Gateway: Administrative Endpoints

[shaneutt]

While building the Kong Gateway API implementation we've noticed a distinction with our underlying Gateway that doesn't appear to have provisions in the spec:

Our Gateway (I mean the actual proxy server in this case) has a distinct administrative API that can be on an entirely separate address from the actual proxy address for application traffic, and in some cases can even be accessible only via localhost to the [Gateway] controller.

This distinction combined with a need to add support for pre-existing gateways (as per @howardjohn's relevant doc) has raised some questions for us. In the operational mode where declaring the Gateway resource actually results in provisioning of the underlying gateway pods (again see John's doc) having a specific place in the status to indicate how that Admin API got provisioned and where it will be listening is very helpful, and having the same thing available in the spec to support using existing gateway deployments is especially helpful. For the purpose of this discussion, let's call these additional endpoints which may have a purposes distinct from application traffic "admin endpoints", and assume they can range from Admin APIs like ours to administrative GUIs, e.t.c.

My question is for implementors: Does anyone else have a similar situation where they have reason to declare these kinds of "admin endpoints" somewhere in the spec/status of a Gateway? Do you solve for this already but another way (maybe annotations)?

My goal is to collect some feedback and possibly create a GEP if there appears to be a more general need for this kind of provision.

[howardjohn]

For us we have a hardcoded port that is always the same and bound to localhost. If there was an API to declare this we wouldn't mind setting it but it's not very important to us. Our interface is largely developer facing though, if we had a nice UI it seems like it would be more useful

[jpeach]

having a specific place in the status to indicate how that Admin API got provisioned and where it will be listening is very helpful, and having the same thing available in the spec to support using existing gateway deployments is especially helpful.

Who is the audience for this infomation? Human cluster operators, tools, controllers?

[shaneutt]

I think a case can be made for all of the above.

[jpeach]

Then I'd recommend exploring publishing the info you need in annotations, then pushing that up to humans in your developer tooling.

[shaneutt]

Yes. I suppose I didn't make this clear before but this is the path that would be taken by default if there's not a strong signal from others that there's a shared need for this amongst implementations. 👍

[youngnick]

For Contour, we recently moved to only exposing the Envoy admin UI and other action-taking things over a domain socket, because users could use ExternalName Services to acceess the localhost in the Envoy container, and so expose the Envoy Admin UI to the world.

For us, the only analogous thing is how the Envoy deployment knows how to connect back to Contour proper's xDS port - but we solve this with convention rather than configurability. Because of how the xDS dance, required TLS etc works, I don't see us changing this any time soon.

[shaneutt]

Good feedback too, thanks 👍

[shaneutt]

We talked about this in today's community meeting. Between the existing comments here in this thread and the conversation today in Zoom it would appear that there is not a shared requirement to think about Gateway objects in terms of the endpoints available on them (specifically including administrative ones as distinct from application traffic ones) widely across implementations at this time (though there appears to be some interest). This lack of need would seem to be potentially due to how rigid the deployment of these Gateways are for most implementations at this early phase in Gateway's life.

We further discussed taking some of the ideas here in this doc and on our conversation today and distilling them along with John's doc on relevant matters into a guide that helps provide a little bit more prescription to implementors on how to solve some of the problems of existing vs. dynamically provisioned.

The resulting follow-up to this discussion is now: #892