The deployment modes that Kuma provides are quite unique in the Service Mesh landscape and have been developed thanks to the guidance of our enterprise users, especially when it comes to the distributed one.
There are two deployment models that can be adopted with Kuma in order to address any Service Mesh use-case, from the simple one running in one zone to the more complex one where multiple Kubernetes or VM zones are involved, or even hybrid universal ones where Kuma runs simultaneously on Kubernetes and VMs.
The two deployments modes are:
- Standalone: Kuma's default deployment model with one control plane (that can be scaled horizontally) and many data planes connecting directly to it.
- Multi-Zone: Kuma's advanced deployment model to support multiple Kubernetes or VM-based zones, or hybrid Service Meshes running on both Kubernetes and VMs combined.
Automatic Connectivity: Running a Service Mesh should be easy and connectivity should be abstracted away, so that when a service wants to consume another service all it needs is the name of the destination service. Kuma achieves this out of the box in both deployment modes with a built-in service discovery and - in the case of the multi-zone mode - with an Ingress resource and Remote CPs.
# Standalone Mode
This is the simplest deployment mode for Kuma, and the default one.
- Control plane: There is one deployment of the control plane that can be scaled horizontally.
- Data planes: The data planes connect to the control plane regardless of where they are being deployed.
- Service Connectivity: Every data plane proxy must be able to connect to every other data plane proxy regardless of where they are being deployed.
This mode implies that we can deploy Kuma and its data plane proxies in a standalone networking topology mode so that the service connectivity from every data plane proxy can be established directly to every other data plane proxy.
Although standalone mode can support complex multi-zone or hybrid deployments (Kubernetes + VMs) as long as the networking requirements are satisfied, typically in most use cases our connectivity cannot be flattened out across multiple zones. Therefore standalone mode is usually a great choice within the context of one zone (ie: within one Kubernetes cluster or one AWS VPC).
For those situations where the standalone deployment mode doesn't satisfy our architecture, Kuma provides a multi-zone mode which is more powerful and provides a greater degree of flexibility in more complex environments.
In order to deploy Kuma in a standalone deployment, the
kuma-cp control plane must be started in
Once Kuma is up and running, data plane proxies can now connect directly to it.
When the mode is not specified, Kuma will always start in
standalone mode by default.
# Multi-Zone Mode
This is a more advanced deployment mode for Kuma that allow us to support service meshes that are running on many zones, including hybrid deployments on both Kubernetes and VMs.
- Control plane: There is one
globalcontrol plane, and many
remotecontrol planes. A global control plane only accepts connections from remote control planes.
- Data planes: The data planes connect to the closest
remotecontrol plane in the same zone. Additionally, we need to start an
ingressdata plane on every zone to have cross-zone communication between data planes in different zones.
- Service Connectivity: Automatically resolved via the built-in DNS resolver that ships with Kuma. When a service wants to consume another service, it will resolve the DNS address of the desired service with Kuma, and Kuma will respond with a Virtual IP address, that corresponds to that service in the Kuma service domain.
We can support multiple isolated service meshes thanks to Kuma's multi-tenancy support, and workloads from both Kubernetes or any other supported Universal environment can participate in the Service Mesh across different regions, clouds, and datacenters while not compromizing the ease of use and still allowing for end-to-end service connectivity.
When running in multi-zone mode, we introduce the notion of a
remote control planes for Kuma:
- Global: this control plane will be used to configure the global Service Mesh policies that we want to apply to our data plane proxies. Data plane proxies cannot connect directly to a global control plane, but can connect to
remotecontrol planes that are being deployed on each underlying zone that we want to include as part of the Service Mesh (can be a Kubernetes cluster, or VM based). Only one deployment of the global control plane is required, and it can be scaled horizontally.
- Remote: we are going to have as many remote control planes as the number of underlying Kubernetes or VM zones that we want to include in a Kuma mesh. Remote control planes will accept connections from data planes that are being started in the same underlying zone, and they will themselves connect to the
globalcontrol plane in order to fetch the service mesh policies that have been configured. Remote control plane policy APIs are read-only and cannot accept Service Mesh policies to be directly configured on them. They can be scaled horizontally within their zone.
In this deployment, a Kuma cluster is made of one global control plane and as many remote control planes as the number of zones that we want to support:
- Zone: A zone identifies a Kubernetes cluster, a VPC, or any other cluster that we want to include in a Kuma service mesh.
In a multi-zone deployment mode, services will be running on multiple platforms, clouds, or Kubernetes clusters (which are identifies as
zones in Kuma). While all of them will be part of a Kuma mesh by connecting their data plane proxies to the local
remote control plane in the same zone, implementing service to service connectivity would be tricky since a source service may not know where a destination service is being hosted at (for instance, in another zone).
To implement easy service connectivity, Kuma ships with:
- DNS Resolver: Kuma provides an out of the box DNS server on every
remotecontrol plane that will be used to resolve service addresses when estabilishing any service-to-service communication. It scales horizontally as we scale the
- Ingress Data Plane: Kuma provides an out of the box
ingressdata plane mode that will be used to enable traffic to enter a zone from another zone. It can be scaled horizontally. Each zone must have an
ingressdata plane deployed.
ingress data plane is specific to internal communication within a mesh and it is not to be considered an API gateway. API gateways are supported via Kuma's gateway mode which can be deployed in addition to
ingress data planes.
The global control plane and the remote control planes communicate with each other via xDS in order to synchronize the resources that are being created to configure Kuma, like policies.
For Kubernetes: The global control plane on Kubernetes must reside on its own Kubernetes cluster, in order to keep the CRDs separate from the ones that the remote control planes will create during the synchronization process.
In order to deploy Kuma in a multi-zone deployment, we must start a
global and as many
remote control planes as the number of zones that we want to support.
# Global control plane
First we start the
global control plane and configure the
remote control planes connectivity.
# Remote control plane
remote control planes in each zone that will be part of the multi-zone Kuma deployment.
remote control plane, you need to assign the zone name for each of them and point it to the Global CP.
# Verify control plane connectivity
When a remote control plane connects to the global control plane, the
Zone resource is created automatically in the global control plane.
You can verify if a remote control plane is connected to the global control plane by inspecting the list of zones in the global control plane GUI (
:5681/gui/#/zones) or by using
kumactl get zones.
Additionally, if you deployed remote control plane with Ingress, it should be visible in the Ingress tab of the GUI. Cross-zone communication between services is only available if Ingress has a public address and public port. Note that on Kubernetes, Kuma automatically tries to pick up the public address and port. Depending on the LB implementation of your Kubernetes provider, you may need to wait a couple of minutes to receive the address.
# Enable mTLS
# Using the multi-zone deployment
To utilize the multi-zone Kuma deployment follow the steps below
The Kuma DNS service format (e.g.
echo-server_kuma-test_svc_1010.mesh) is a composition of Kubernetes Service Name (
kuma-test), a fixed string (
svc), the service port (
1010). The service is resolvable in the DNS zone
the Kuma DNS service is hooked.
# Deleting a Zone
To delete a
Zone we must first shut down the corresponding Kuma Remote CP instances. As long as the Remote CP is running this will not be possible, and Kuma will return a validation error like:
zone: unable to delete Zone, Remote CP is still connected, please shut it down first
When the Remote CP is fully disconnected and shut down, then the
Zone can be deleted. All corresponding resources (like
DataplaneInsight) will be deleted automatically as well.
# Disabling zone
In order to disable routing traffic to a specific
Zone, we can disable the
Zone via the
type: Zone name: zone-1 spec: enabled: true
Changing this value to
enabled: false will allow the user to exclude the zone's
Ingress from all other zones - and by doing so - preventing traffic from being able to enter the
Zone that has been disabled will show up as "Offline" in the GUI and CLI