Any plans to add self-hosted LLM options to AIGateway K8s custom resource?

[davidxia]

Problem Statement

I want to configure self-hosted LLM backends with Kong. I deployed Kong with the Gateway Operator.

I see the AIGateway K8s custom resource has .spec.largeLanguageModels.cloudHosted as a field. The description states

This is currently a required field, requiring at least one cloud-hosted LLM be specified, however in future iterations we may add other hosting options such as self-hosted LLMs as separate fields.

Are there any plans to add self-hosted LLM options to AIGateway? Are there other ways to configure self-hosted LLM backends with Kong deployed with Gateway Operator?

Acceptance Criteria

• I can configure self-hosted LLM backends with the AIGateway CR