Kubernetes Gateway API
Kuma supports Kubernetes Gateway API for configuring built-in gateway as well as traffic routing using the experimental GAMMA routing spec.
Installation
Kubernetes doesn’t include Gateway API CRDs, install them from the standard channel CRD bundle.
Gateways
-
Install the counter demo.
kumactl install demo --without-gateway | kubectl apply -f -
-
Add a
Gateway
.The
Gateway
resource represents the proxy instance that handles traffic for a set of Gateway API routes.Every
Gateway
refers to aGatewayClass
. TheGatewayClass
represents the class ofGateway
, in this case Kuma’s builtin edge gateway, and points to a controller that should manage theseGateways
. It can also hold additional configuration.For Helm and
kumactl
installations, aGatewayClass
namedkuma
is automatically installed if the Gateway API CRDs are present.apiVersion: gateway.networking.k8s.io/v1 kind: Gateway metadata: name: kuma namespace: kuma-demo spec: gatewayClassName: kuma listeners: - name: proxy port: 8080 protocol: HTTP
When a user applies a
Gateway
resource, Kuma automatically creates aDeployment
of built-in gateways with a correspondingService
.kubectl get pods -n kuma-demo
NAME READY STATUS RESTARTS AGE redis-59c9d56fc-6gcbc 2/2 Running 0 2m8s demo-app-5845d6447b-v7npw 2/2 Running 0 2m8s kuma-4j6wr-58998b5576-25wl6 1/1 Running 0 30s
kubectl get svc -n kuma-demo
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE redis ClusterIP 10.43.223.223 <none> 6379/TCP 3m27s demo-app ClusterIP 10.43.216.203 <none> 5000/TCP 3m27s kuma-pfh4s LoadBalancer 10.43.122.93 172.20.0.3 8080:30627/TCP 87s
The
Gateway
is now accessible using the external address172.20.0.3:8080
. -
Add an
HTTPRoute
.HTTPRoute
resources contain a set of matching criteria for HTTP requests and upstreamServices
to route those requests to.apiVersion: gateway.networking.k8s.io/v1 kind: HTTPRoute metadata: name: echo namespace: kuma-demo spec: parentRefs: - group: gateway.networking.k8s.io kind: Gateway name: kuma namespace: kuma-demo rules: - backendRefs: - kind: Service name: demo-app port: 5000 weight: 1 matches: - path: type: PathPrefix value: /
After creating an
HTTPRoute
, accessing/
forwards a request to the demo app:curl 172.20.0.3:8080/ -i
HTTP/1.1 200 OK x-powered-by: Express accept-ranges: bytes cache-control: public, max-age=0 last-modified: Tue, 20 Oct 2020 17:16:41 GMT etag: W/"2b91-175470350a8" content-type: text/html; charset=UTF-8 content-length: 11153 date: Fri, 18 Mar 2022 11:33:29 GMT x-envoy-upstream-service-time: 2 server: Kuma Gateway <html> <head> ...
TLS termination
Gateway API supports TLS termination by using standard kubernetes.io/tls
Secrets.
Here is an example
apiVersion: v1
kind: Secret
metadata:
name: secret-tls
namespace: kuma-demo
type: kubernetes.io/tls
data:
tls.crt: "MIIEOzCCAyO..." # redacted
tls.key: "MIIEowIBAAKC..." # redacted
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: kuma
namespace: kuma-demo
spec:
gatewayClassName: kuma
listeners:
- name: proxy
port: 8080
hostname: test.kuma.io
protocol: HTTPS
tls:
certificateRefs:
- name: secret-tls
Under the hood, Kuma CP copies the Secret
to kuma-system
namespace and converts it to
Kuma secret
.
It tracks all the changes to the secret and deletes it upon deletion of the original secret.
Customization
Gateway API provides the parametersRef
field on GatewayClass.spec
to provide additional, implementation-specific configuration to Gateways
.
When using Gateway API with Kuma, you can refer to a MeshGatewayConfig
resource:
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
name: kuma
spec:
controllerName: gateways.kuma.io/controller
parametersRef:
kind: MeshGatewayConfig
group: kuma.io
name: kuma
This resource has the same structure as the MeshGatewayInstance
resource
except that the tags
field is optional.
With a MeshGatewayConfig
you can then customize
the generated Service
and Deployment
resources.
Multi-mesh
You can specify a Mesh
for Gateway
and HTTPRoute
resources
by setting the kuma.io/mesh
annotation
Note that HTTPRoutes
must also have the annotation to reference a
Gateway
from a non-default Mesh
.
Cross-mesh
Cross-mesh gateways are supported with Gateway API.
You’ll just need to create a corresponding GatewayClass
pointing to a MeshGatewayConfig
that
sets crossMesh: true
:
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
name: kuma-cross-mesh
spec:
controllerName: gateways.kuma.io/controller
parametersRef:
group: kuma.io
kind: MeshGatewayConfig
name: default-cross-mesh
---
apiVersion: kuma.io/v1alpha1
kind: MeshGatewayConfig
metadata:
name: default-cross-mesh
spec:
crossMesh: true
and then reference it in your Gateway
:
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: kuma
namespace: default
spec:
gatewayClassName: kuma-cross-mesh
listeners:
- name: proxy
port: 8080
protocol: HTTP
Multi-zone Deployments
The Gateway API supports multi-zone deployments, but with some limitations:
-
Gateway API resources like
Gateway
,ReferenceGrant
, andHTTPRoute
must be created in non-global zones. -
Only services deployed within the same Kubernetes cluster, such as the
HTTPRoute
, can be referenced viabackendRef
.Important Note: This limitation exist because, Kuma currently only allows referencing as
backendRefs
Kubernetes Services.This is a temporary limitation. We’re actively working on extending
backendRef
to support Kuma’sMeshServices
. Once this feature is complete, you’ll be able to reference services across different clusters within your mesh.To better visualize this limitation here’s an example scenario that describes how you could configure multi-zone deployments with the Gateway API. In this example, you have the following resources:
-
Two zones (
zone-1
andzone-2
) in separate Kubernetes clusters -
Gateway with listener on port
8080
deployed inzone-1
-
Two services:
-
A service named
backend
deployed in each zone -
A service named
db
deployed only inzone-2
-
flowchart TD subgraph c2["k8s-cluster-2"] subgraph z2["zone-2"] subgraph c1z2s1["Service"] b2(backend) end subgraph c1z2s2["Service"] db(db) end end end subgraph c1["k8s-cluster-1"] subgraph z1["zone-1"] subgraph Gateway listener(:8080) end subgraph Service b1(backend) end end end
If you deploy multi-zone with Gateway API, the following will occur:
- If you create an
HTTPRoute
with abackendRef
targeting thebackend
service ink8s-cluster-1
, it will only route traffic to thebackend
service ink8s-cluster-1
.
flowchart TD subgraph c2["k8s-cluster-2"] subgraph z2["zone-2"] subgraph c1z2s1["Service"] backend2(backend) end subgraph c1z2s2["Service"] db(db) end end end subgraph c1["k8s-cluster-1"] subgraph z1["zone-1"] subgraph Service backend1(backend) end subgraph Gateway listener(:8080) end subgraph HTTPRoute route1(/) end route1--"❌"-->backend2 linkStyle 0 stroke:red,color:red,stroke-dasharray: 5 5; route1-->backend1 listener-->route1 end end
- Similarly, if you create an
HTTPRoute
with abackendRef
pointing to thedb
service ink8s-cluster-1
, it will result in aHTTPRoute
with aResolvedRefs
status condition ofBackendNotFound
because servicedb
is not present ink8s-cluster-1
.
flowchart TD subgraph c2["k8s-cluster-2"] subgraph z2["zone-2"] subgraph c1z2s1["Service"] backend2(backend) end subgraph c1z2s2["Service"] db(db) end end end subgraph c1["k8s-cluster-1"] subgraph z1["zone-1"] subgraph Gateway listener(:8080) end subgraph HTTPRoute route1(/) end subgraph Service backend1(backend) end route1--"❌"-->db linkStyle 0 stroke:red,color:red,stroke-dasharray: 5 5; listener-->route1 end end
-
Service to service routing
Kuma also supports routing between services with
HTTPRoute
in conformance with
the GAMMA specifications.
GAMMA is a Gateway API subproject focused on mesh implementations of Gateway API and extending the Gateway API resources to mesh use cases.
The key feature of HTTPRoute
for mesh routing is specifying a Kubernetes
Service
as the parentRef
, as opposed to a Gateway
.
All requests to this Service
are then filtered and routed as specified in the
HTTPRoute
.
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: canary-demo-app
namespace: kuma-demo
spec:
parentRefs:
- name: demo-app
port: 5000
kind: Service
rules:
- backendRefs:
- name: demo-app-v1
port: 5000
- name: demo-app-v2
port: 5000
The namespace of the HTTPRoute
is key. If the route’s namespace and the
parentRef
’s namespace are identical, Kuma applies
the route to requests from all workloads.
If the route’s namespace differs from its parentRef
’s namespace,
the HTTPRoute
applies only to requests
from workloads in the route’s namespace.
Remember to tag your Service
ports with appProtocol: http
to use
them in an HTTPRoute
!
Because of how Kuma maps resources at the moment,
the combination of the HTTPRoute
s name and namespace and
the parent Service
name and namespace must be no more than 249 characters.
How it works
Kuma includes controllers that reconcile Gateway API CRDs and convert them into the corresponding Kuma CRDs.
This is why in the GUI, Kuma MeshGateways
/MeshHTTPRoutes
/MeshTCPRoutes
are visible and not Kubernetes Gateway API resources.
Kubernetes Gateway API resources serve as the source of truth for Kuma gateways and routes. Any edits to the corresponding Kuma resources are overwritten.