Careful!

You are browsing documentation for a version of Kuma that is not the latest release.

Configuring your Mesh and multi-tenancy

This resource describes a very important concept in Kuma, and that is the ability of creating multiple isolated service meshes within the same Kuma cluster which in turn make Kuma a very simple and easy project to operate in environments where more than one mesh is required based on security, segmentation or governance requirements.

Typically, we would want to create a Mesh per line of business, per team, per application or per environment or for any other reason. Typically multiple meshes are being created so that a service mesh can be adopted by an organization with a gradual roll-out that doesn’t require all the teams and their applications to coordinate with each other, or as an extra layer of security and segmentation for our services so that - for example - policies applied to one Mesh do not affect another Mesh.

Mesh is the parent resource of every other resource in Kuma, including:

In order to use Kuma at least one Mesh must exist, and there is no limit to the number of Meshes that can be created. When a data plane proxy connects to the control plane (kuma-cp) it specifies to what Mesh resource it belongs: a data plane proxy can only belong to one Mesh at a time.

When starting a new Kuma cluster from scratch a default Mesh is being created automatically.

Besides the ability of being able to create virtual service mesh, a Mesh resource will also be used for:

Mutual TLS, to secure and encrypt our service traffic and assign an identity to the data plane proxies within the Mesh.
Traffic Metrics
Traffic Trace
Zone Egress, to setup if ZoneEgress should be used for cross zone and external service communication.
Non-mesh traffic, to setup if passthrough mode should be used for the non-mesh traffic.

To support cross-mesh communication an intermediate API Gateway must be used. Kuma checkout Kuma’s builtin gateway to set this up.

Usage

The easiest way to create a Mesh is to specify its name. The name of a Mesh must be unique.

Kubernetes
Universal

apiVersion: kuma.io/v1alpha1
kind: Mesh
metadata:
  name: default

We will apply the configuration with kubectl apply -f [..].

Creating resources in a Mesh

It is possible to determine to what Mesh other resources belong to in the following ways.

Data plane proxies

Every time we start a data plane proxy, we need to specify to what Mesh it belongs, this can be done in the following way:

Kubernetes
Universal

By using the kuma.io/mesh annotation in a Deployment, like:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: example-app
  namespace: kuma-example
spec:
  ...
  template:
    metadata:
      ...
      annotations:
        # indicate to Kuma what is the Mesh that the data plane proxy belongs to
        kuma.io/mesh: default
    spec:
      containers:
        ...

A Mesh may span multiple Kubernetes namespaces. Any Kuma resource in the cluster which specifies a particular Mesh will be part of that Mesh.

By using the -m or --mesh argument when running kuma-dp, for example:

kuma-dp run \
  --name=backend-1 \
  --mesh=default \
  --cp-address=https://127.0.0.1:5678 \
  --dataplane-token-file=/tmp/kuma-dp-backend-1-token

You can control which data plane proxies are allowed to join the mesh using mesh constraints.

Policies

When creating new Policies we also must specify to what Mesh they belong. This can be done in the following way:

By using the mesh property, like:

apiVersion: kuma.io/v1alpha1
kind: TrafficRoute
mesh: default # indicate to Kuma what is the Mesh that the resource belongs to
metadata:
  name: route-1
spec:
  ...

Kuma consumes all Policies on the cluster and joins each to an individual Mesh, identified by this property.

By using the kuma.io/mesh label, like:

apiVersion: kuma.io/v1alpha1
kind: MeshHTTPRoute
metadata:
  name: route-1
  namespace: kuma-system
  labels:
    kuma.io/mesh: default # indicate to Kuma what is the Mesh that the resource belongs to
spec:
  ...

Kuma consumes all Policies on the cluster and joins each to an individual Mesh, identified by this property.

Skipping default resource creation

By default, to help users get started we create the following default policies:

        
          Kubernetes
        
      

        
          Universal
        
      

    
      apiVersion: kuma.io/v1alpha1
kind: MeshTimeout
metadata:
  name: mesh-gateways-timeout-all-default
  namespace: kuma-system
  labels:
    kuma.io/mesh: default
spec:
  targetRef:
    kind: Mesh
    proxyTypes:
    - Gateway
  to:
  - targetRef:
      kind: Mesh
    default:
      idleTimeout: 1h
      http:
        streamIdleTimeout: 5s
  from:
  - targetRef:
      kind: Mesh
    default:
      idleTimeout: 5m
      http:
        streamIdleTimeout: 5s
        requestHeadersTimeout: 500ms
---
apiVersion: kuma.io/v1alpha1
kind: MeshTimeout
metadata:
  name: mesh-timeout-all-default
  namespace: kuma-system
  labels:
    kuma.io/mesh: default
spec:
  targetRef:
    kind: Mesh
    proxyTypes:
    - Sidecar
  to:
  - targetRef:
      kind: Mesh
    default:
      connectionTimeout: 5s
      idleTimeout: 1h
      http:
        requestTimeout: 15s
        streamIdleTimeout: 30m
  from:
  - targetRef:
      kind: Mesh
    default:
      connectionTimeout: 10s
      idleTimeout: 2h
      http:
        requestTimeout: 0s
        streamIdleTimeout: 1h
        maxStreamDuration: 0s
---
apiVersion: kuma.io/v1alpha1
kind: MeshRetry
metadata:
  name: mesh-retry-all-default
  namespace: kuma-system
  labels:
    kuma.io/mesh: default
spec:
  targetRef:
    kind: Mesh
  to:
  - targetRef:
      kind: Mesh
    default:
      tcp:
        maxConnectAttempt: 5
      http:
        numRetries: 5
        perTryTimeout: 16s
        backOff:
          baseInterval: 25ms
          maxInterval: 250ms
      grpc:
        numRetries: 5
        perTryTimeout: 16s
        backOff:
          baseInterval: 25ms
          maxInterval: 250ms
---
apiVersion: kuma.io/v1alpha1
kind: MeshCircuitBreaker
metadata:
  name: mesh-circuit-breaker-all-default
  namespace: kuma-system
  labels:
    kuma.io/mesh: default
spec:
  targetRef:
    kind: Mesh
  to:
  - targetRef:
      kind: Mesh
    default:
      connectionLimits:
        maxConnections: 1024
        maxPendingRequests: 1024
        maxRetries: 3
        maxRequests: 1024

    
      type: MeshTimeout
mesh: default
name: mesh-gateways-timeout-all-default
spec:
  targetRef:
    kind: Mesh
    proxyTypes:
    - Gateway
  to:
  - targetRef:
      kind: Mesh
    default:
      idleTimeout: 1h
      http:
        streamIdleTimeout: 5s
  from:
  - targetRef:
      kind: Mesh
    default:
      idleTimeout: 5m
      http:
        streamIdleTimeout: 5s
        requestHeadersTimeout: 500ms
---
type: MeshTimeout
mesh: default
name: mesh-timeout-all-default
spec:
  targetRef:
    kind: Mesh
    proxyTypes:
    - Sidecar
  to:
  - targetRef:
      kind: Mesh
    default:
      connectionTimeout: 5s
      idleTimeout: 1h
      http:
        requestTimeout: 15s
        streamIdleTimeout: 30m
  from:
  - targetRef:
      kind: Mesh
    default:
      connectionTimeout: 10s
      idleTimeout: 2h
      http:
        requestTimeout: 0s
        streamIdleTimeout: 1h
        maxStreamDuration: 0s
---
type: MeshRetry
mesh: default
name: mesh-retry-all-default
spec:
  targetRef:
    kind: Mesh
  to:
  - targetRef:
      kind: Mesh
    default:
      tcp:
        maxConnectAttempt: 5
      http:
        numRetries: 5
        perTryTimeout: 16s
        backOff:
          baseInterval: 25ms
          maxInterval: 250ms
      grpc:
        numRetries: 5
        perTryTimeout: 16s
        backOff:
          baseInterval: 25ms
          maxInterval: 250ms
---
type: MeshCircuitBreaker
mesh: default
name: mesh-circuit-breaker-all-default
spec:
  targetRef:
    kind: Mesh
  to:
  - targetRef:
      kind: Mesh
    default:
      connectionLimits:
        maxConnections: 1024
        maxPendingRequests: 1024
        maxRetries: 3
        maxRequests: 1024

    
  

If you want to not have these policies be added on creation of the mesh set the configuration: skipCreatingInitialPolicies:

Kubernetes
Universal

apiVersion: kuma.io/v1alpha1
kind: Mesh
metadata:
  name: default
spec:
  skipCreatingInitialPolicies: ['*']

You can also skip creating the default mesh by setting the control-plane configuration: KUMA_DEFAULTS_SKIP_MESH_CREATION=true.

All options

$schema: http://json-schema.org/draft-04/schema#

$ref: #/definitions/Mesh

definitions

Mesh

## Mesh
Mesh defines configuration of a single mesh.
Type: object
This schema accepts additional properties.
Properties
- mtls
  - mTLS settings. +optional
  - Type: object
  - $ref: #/definitions/kuma.mesh.v1alpha1.Mesh.Mtls
  - This schema accepts additional properties.
  - Properties
- tracing
  - Tracing settings. +optional
  - Type: object
  - $ref: #/definitions/kuma.mesh.v1alpha1.Tracing
  - This schema accepts additional properties.
  - Properties
- logging
  - Logging settings. +optional
  - Type: object
  - $ref: #/definitions/kuma.mesh.v1alpha1.Logging
  - This schema accepts additional properties.
  - Properties
- metrics
  - Configuration for metrics collected and exposed by dataplanes. Settings defined here become defaults for every dataplane in a given Mesh. Additionally, it is also possible to further customize this configuration for each dataplane individually using Dataplane resource. +optional
  - Type: object
  - $ref: #/definitions/kuma.mesh.v1alpha1.Metrics
  - This schema accepts additional properties.
  - Properties
- networking
  - Networking settings of the mesh
  - Type: object
  - $ref: #/definitions/kuma.mesh.v1alpha1.Networking
  - This schema accepts additional properties.
  - Properties
- routing
  - Routing settings of the mesh
  - Type: object
  - $ref: #/definitions/kuma.mesh.v1alpha1.Routing
  - This schema accepts additional properties.
  - Properties
- constraints
  - Constraints that applies to the mesh and its entities
  - Type: object
  - $ref: #/definitions/kuma.mesh.v1alpha1.Mesh.Constraints
  - This schema accepts additional properties.
  - Properties
- skipCreatingInitialPolicies
  - List of policies to skip creating by default when the mesh is created. e.g. TrafficPermission, MeshRetry, etc. An '' can be used to skip all policies.*
  - Type: array
    - Items
    - Type: string kuma.mesh.v1alpha1.CertificateAuthorityBackend
## Certificate Authority Backend
CertificateAuthorityBackend defines Certificate Authority backend
Type: object
This schema accepts additional properties.
Properties
- name
  - Name of the backend
  - Type: string
- type
  - Type of the backend. Has to be one of the loaded plugins (Kuma ships with builtin and provided)
  - Type: string
- dpCert
  - Dataplane certificate settings
  - Type: object
  - $ref: #/definitions/kuma.mesh.v1alpha1.CertificateAuthorityBackend.DpCert
  - This schema accepts additional properties.
  - Properties
- conf
  - Configuration of the backend
  - Type: object
  - This schema accepts additional properties.
  - Properties
- mode
  - #### Mode
  - The value is restricted to the following:
    1. "STRICT"
    2. 0
    3. "PERMISSIVE"
    4. 1
- rootChain
  - Type: object
  - $ref: #/definitions/kuma.mesh.v1alpha1.CertificateAuthorityBackend.RootChain
  - This schema accepts additional properties.
  - Properties kuma.mesh.v1alpha1.CertificateAuthorityBackend.DpCert
## Dp Cert
DpCert defines settings for certificates generated for Dataplanes
Type: object
This schema accepts additional properties.
Properties
- rotation
  - Rotation settings
  - Type: object
  - $ref: #/definitions/kuma.mesh.v1alpha1.CertificateAuthorityBackend.DpCert.Rotation
  - This schema accepts additional properties.
  - Properties
- requestTimeout
  - Timeout on request to CA for DP certificate generation and retrieval
  - Type: string
  - String format must be a "regex"
  - The value must match this pattern: ^([0-9]+\.?[0-9]*|\.[0-9]+)s$ kuma.mesh.v1alpha1.CertificateAuthorityBackend.DpCert.Rotation
## Rotation
Rotation defines rotation settings for Dataplane certificate
Type: object
This schema accepts additional properties.
Properties
- expiration
  - Time after which generated certificate for Dataplane will expire
  - Type: string kuma.mesh.v1alpha1.CertificateAuthorityBackend.RootChain
## Root Chain
RootChain defines settings related to CA root certificate chain.
Type: object
This schema accepts additional properties.
Properties
- requestTimeout
  - Timeout on request for to CA for root certificate chain. If not specified, defaults to 10s.
  - Type: string
  - String format must be a "regex"
  - The value must match this pattern: ^([0-9]+\.?[0-9]*|\.[0-9]+)s$ kuma.mesh.v1alpha1.Logging
## Logging
Type: object
This schema accepts additional properties.
Properties
- defaultBackend
  - Name of the default backend
  - Type: string
- backends
  - List of available logging backends
  - Type: array
    - Items
    - $ref: #/definitions/kuma.mesh.v1alpha1.LoggingBackend kuma.mesh.v1alpha1.LoggingBackend
## Logging Backend
LoggingBackend defines logging backend available to mesh. Backends can be used in TrafficLog rules.
Type: object
This schema accepts additional properties.
Properties
- name
  - Name of the backend, can be then used in Mesh.logging.defaultBackend or in TrafficLogging
  - Type: string
- format
  - Format of access logs. Placeholders available on https://www.envoyproxy.io/docs/envoy/latest/configuration/observability/access_log
  - Type: string
- type
  - Type of the backend (Kuma ships with 'tcp' and 'file')
  - Type: string
- conf
  - Configuration of the backend
  - Type: object
  - This schema accepts additional properties.
  - Properties kuma.mesh.v1alpha1.Mesh.Constraints
## Constraints
Constraints to apply to the mesh and its entities
Type: object
This schema accepts additional properties.
Properties
- dataplaneProxy
  - DataplaneProxyMembership defines a set of requirements for data plane proxies to be a member of the mesh.
  - Type: object
  - $ref: #/definitions/kuma.mesh.v1alpha1.Mesh.DataplaneProxyConstraints
  - This schema accepts additional properties.
  - Properties kuma.mesh.v1alpha1.Mesh.DataplaneProxyConstraints
## Dataplane Proxy Constraints
Type: object
This schema accepts additional properties.
Properties
- requirements
  - Requirements defines a set of requirements that data plane proxies must fulfill in order to join the mesh. A data plane proxy must fulfill at least one requirement in order to join the mesh. Empty list of allowed requirements means that any proxy that is not explicitly denied can join.
  - Type: array
    - Items
    - $ref: #/definitions/kuma.mesh.v1alpha1.Mesh.DataplaneProxyConstraints.Rules
- restrictions
  - Restrictions defines a set of restrictions that data plane proxies cannot fulfill in order to join the mesh. A data plane proxy cannot fulfill any requirement in order to join the mesh. Restrictions takes precedence over requirements.
  - Type: array
    - Items
    - $ref: #/definitions/kuma.mesh.v1alpha1.Mesh.DataplaneProxyConstraints.Rules kuma.mesh.v1alpha1.Mesh.DataplaneProxyConstraints.Rules
## Rules
Rules defines a set of rules for data plane proxies to be member of the mesh.
Type: object
This schema accepts additional properties.
Properties
- tags
  - Tags defines set of required tags. You can specify '' in value to require non empty value of tag*
  - Type: object
  - This schema accepts additional properties.
  - Properties kuma.mesh.v1alpha1.Mesh.Mtls
## Mtls
mTLS settings of a Mesh.
Type: object
This schema accepts additional properties.
Properties
- enabledBackend
  - Name of the enabled backend
  - Type: string
- backends
  - List of available Certificate Authority backends
  - Type: array
    - Items
    - $ref: #/definitions/kuma.mesh.v1alpha1.CertificateAuthorityBackend
- skipValidation
  - If enabled, skips CA validation.
  - Type: boolean kuma.mesh.v1alpha1.Metrics
## Metrics
Metrics defines configuration for metrics that should be collected and exposed by dataplanes.
Type: object
This schema accepts additional properties.
Properties
- enabledBackend
  - Name of the enabled backend
  - Type: string
- backends
  - List of available Metrics backends
  - Type: array
    - Items
    - $ref: #/definitions/kuma.mesh.v1alpha1.MetricsBackend kuma.mesh.v1alpha1.MetricsBackend
## Metrics Backend
MetricsBackend defines metric backends
Type: object
This schema accepts additional properties.
Properties
- name
  - Name of the backend, can be then used in Mesh.metrics.enabledBackend
  - Type: string
- type
  - Type of the backend (Kuma ships with 'prometheus')
  - Type: string
- conf
  - Configuration of the backend
  - Type: object
  - This schema accepts additional properties.
  - Properties kuma.mesh.v1alpha1.Networking
## Networking
Networking defines the networking configuration of the mesh
Type: object
This schema accepts additional properties.
Properties
- outbound
  - Outbound settings
  - Type: object
  - $ref: #/definitions/kuma.mesh.v1alpha1.Networking.Outbound
  - This schema accepts additional properties.
  - Properties kuma.mesh.v1alpha1.Networking.Outbound
## Outbound
Outbound describes the common mesh outbound settings
Type: object
This schema accepts additional properties.
Properties
- passthrough
  - Control the passthrough cluster
  - Type: boolean kuma.mesh.v1alpha1.Routing
## Routing
Routing defines configuration for the routing in the mesh
Type: object
This schema accepts additional properties.
Properties
- localityAwareLoadBalancing
  - Enable the Locality Aware Load Balancing
  - Type: boolean
- zoneEgress
  - Enable routing traffic to services in other zone or external services through ZoneEgress. Default: false
  - Type: boolean kuma.mesh.v1alpha1.Tracing
## Tracing
Tracing defines tracing configuration of the mesh.
Type: object
This schema accepts additional properties.
Properties
- defaultBackend
  - Name of the default backend
  - Type: string
- backends
  - List of available tracing backends
  - Type: array
    - Items
    - $ref: #/definitions/kuma.mesh.v1alpha1.TracingBackend kuma.mesh.v1alpha1.TracingBackend
## Tracing Backend
TracingBackend defines tracing backend available to mesh. Backends can be used in TrafficTrace rules.
Type: object
This schema accepts additional properties.
Properties
- name
  - Name of the backend, can be then used in Mesh.tracing.defaultBackend or in TrafficTrace
  - Type: string
- sampling
  - Percentage of traces that will be sent to the backend (range 0.0 - 100.0). Empty value defaults to 100.0%
  - Type: number
- type
  - Type of the backend (Kuma ships with 'zipkin')
  - Type: string
- conf
  - Configuration of the backend
  - Type: object
  - This schema accepts additional properties.
  - Properties

Generated with json-schema-md-doc Fri Apr 04 2025 06:27:00 GMT+0000 (Coordinated Universal Time)