Deploying kgateway as an AI Gateway

In this hands-on lab, learn how to configure and deploy an AI Gateway using kgateway to proxy requests to LLMs from within your Kubernetes cluster.

You will:

• Enable the AI extension: Update the kgateway Helm release to activate AI capabilities via the AI extension.
• Define AI Gateway configuration: Use the GatewayParameters resource to specify settings for an AI Gateway, including setting it to ClusterIP for internal traffic routing.
• Provision the AI Gateway: Deploy a Gateway resource with a listener for LLM traffic, and verify it’s up and running.

By the end of this lab, you'll have a functioning AI Gateway ready to route internal requests to large language models.