Using the Kubernetes Gateway API in traifik

Gateway API (formerly called Service API) is an open source project managed by SIG-NETWORK community. The project address is: https://gateway-api.sigs.k8s.io/ . The main reason is that the Ingress resource object can not meet the network requirements well. In many scenarios, the Ingress controller needs to define annotations or crd to expand its functions, which is very unfavorable to the use of standards and support. The newly launched gateway API aims to enhance the service network through an extensible role-oriented interface.

Gateway API is a collection of API resources in Kubernetes, including GatewayClass, gateway, HTTPRoute, tcprute, Service, etc. these resources jointly build models for various network use cases.

The improvement of Gateway API has many better designs than the current Ingress resource object:

Role oriented - the Gateway consists of various API resources, which are modeled according to the role of using and configuring Kubernetes service network.
Generality - like Ingress, it is a general specification with many implementations, and the Gateway API is a specification standard designed to be supported by many implementations.
More expressive - Gateway API resources support core functions such as Header based matching and traffic weight. These functions can only be realized through custom annotations in progress.
Extensibility - the Gateway API allows custom resources to be linked to various layers of the API, which allows finer customization in the appropriate place of the API structure.

There are other noteworthy features:

GatewayClasses - GatewayClasses formalize the types of load balancing implementations that make it easy for users to understand what capabilities can be obtained through Kubernetes resources.
Shared gateway and cross namespace support - they allow the sharing of load balancers and VIP s, and allow independent routing resources to be bound to the same gateway, which enables teams to safely share (including cross namespace) infrastructure without direct coordination.
Normalized routing and backend - the Gateway API supports typed routing resources and different types of backend, which enables the API to flexibly support various protocols (such as HTTP and gRPC) and various backend services (such as Kubernetes Service, bucket or function).

Role oriented design

Whether roads, power, data centers or Kubernetes clusters, infrastructure is built for sharing. However, shared infrastructure provides a common challenge, that is, how to provide flexibility for infrastructure users while being controlled by the owner.

Kubereway achieves this goal through the centralized design of role-oriented API and network control. It allows shared network infrastructure (hardware load balancer, cloud network, cluster hosted agent, etc.) to be used by many different teams, all of which are subject to various policies and constraints set by cluster operation and maintenance. The following example shows how it works in practice.

A cluster operation and maintenance personnel has created a Gateway resource based on the Gateway class. The Gateway deploys or configures the basic network resources it represents. The cluster operation and maintenance personnel and specific teams must communicate what can be attached to the Gateway to expose their applications. Centralized policies, such as TLS, can be enforced by cluster O & M on the Gateway. At the same time, Store and Site applications run in their own namespace, but attach their routes to the same shared Gateway, allowing them to control their routing logic independently.

This design of separation of concerns can enable different teams to manage their own traffic, while leaving the centralized strategy and control to the cluster operation and maintenance.

concept

There are three roles involved in the whole Gateway API: infrastructure provider, Cluster Administrator and application developer. In some scenarios, the role of application administrator may also be involved. Three main resource models are defined in the Gateway API: Gateway class, Gateway and Route.

GatewayClass

GatewayClass defines a set of gateways that share the same configuration and actions. Each gateway class is handled by a controller and is a cluster wide resource. At least one gateway class must be defined.

This is similar to the IngressClass of Ingress. In Ingress v1beta1, the Ingress class annotation is similar to the gateway class. In Ingress V1, the closest is the IngressClass resource object.

Gateway

Gateway gateway describes how to convert traffic into services in the cluster, that is, it defines a request to convert traffic from places where Kubernetes is not known to services in the cluster. For example, traffic sent to Kubernetes service by cloud load balancer, intra cluster agent or external hardware load balancer.

It defines the request for specific load balancer configuration, which implements the configuration and behavior specification of GatewayClass. This resource can be created directly by the administrator or by the controller handling GatewayClass.

Gateway can be attached to one or more route references, which are used to direct a subset of traffic to specific services.

Route resource

Routing resources define specific rules for mapping requests from gateways to Kubernetes services.

Starting from v1alpha2, the API contains four Route resource types. For other undefined protocols, it is encouraged to adopt custom Route types for specific implementations. Of course, new Route types may be added in the future.

HTTPRoute

HTTPRoute is applicable to HTTP or HTTPS connections. It is applicable to scenarios where we want to check HTTP requests and route or modify them with HTTP requests, such as routing with HTTP Headers or modifying them during the request process.

TLSRoute

TLSRoute is used for TLS connections and distinguished by SNI. It is applicable to places where SNI is the main routing method. It is not interested in the properties of higher-level protocols such as HTTP. The connected byte stream is proxy to the back end without any inspection.

Tcprune and udprune

Tcprune (and udprune) is designed to map one or more ports to a single backend. In this case, there is no discriminator that can be used to select different backend of the same port, so each tcprute needs a different port on the listener. You can use TLS. In this case, the unencrypted byte stream will be delivered to the back end. Of course, you can also not use TLS, so that the encrypted byte stream will be delivered to the back end.

combination

The combination of GatewayClass, Gateway, xRoute and Service defines an implementable load balancer. The following figure illustrates the relationship between different resources:

The typical client / gateway API request process of the gateway implemented by reverse proxy is as follows:

1. Client to http://foo.example.com Make a request
2. DNS resolves the domain name to the Gateway address
3. The reverse proxy receives the request on the listener and uses the Host Header to match the HTTPRoute
4. (optional) the reverse proxy can route according to the matching rules of HTTPRoute
5. (optional) the reverse proxy can modify the request according to the filtering rules of HTTPRoute, that is, add or delete headers
6. Finally, the reverse proxy forwards the request to one or more objects in the cluster, namely services, according to the forwardTo rule of HTTP route.

realization

At present, there are many controller implementation schemes of Gateway API, such as Contour, Google Kubernetes Engine, Istio, traifik and so on. Next, we take Traefik as an example to test. However, it should be noted that Traefik is currently implemented based on the v1alpha1 specification, which may be slightly different from some of the concepts mentioned above.

To use the Gateway API in Traefik, first we need to manually install the CRDs of the Gateway API, which can be installed by using the following commands. This will install CRDs including Gateway class, Gateway, HTTPRoute, tcprute and so on:

➜ kubectl kustomize "github.com/kubernetes-sigs/gateway-api/config/crd?ref=v0.3.0" \
| kubectl apply -f -

Then we need to open the kubernetesgateway Provider in traifik, and also define it based on the Helm Chart package in the previous chapter of traifik, and set experimental kubernetesGateway. Enabled = true, the complete Values file is as follows:

# ci/deployment-prod.yaml

# Enable experimental features
experimental:
  kubernetesGateway:  # Enable gateway api support
    enabled: true

providers:
  kubernetesCRD:
    enabled: true
    allowCrossNamespace: true  # Allow cross namespace
    allowExternalNameServices: true  # Allow services with ExternalName

  kubernetesIngress:
    enabled: true
    allowExternalNameServices: true

# ......
# Other ignore

Then update traifik with the following command:

➜ helm upgrade --install traefik ./traefik -f ./traefik/ci/deployment-prod.yaml --namespace kube-system

After the update, you can go to the Dashboard of Traefik to check whether the KubernetesGateway Provider has been enabled:

Under normal circumstances, after successful activation, traifik will also create a default Gateway class resource object and Gateway instance:

➜ kubectl get gatewayclass
NAME      CONTROLLER                      AGE
traefik   traefik.io/gateway-controller   4m13s
➜ kubectl get gatewayclass traefik -o yaml
apiVersion: networking.x-k8s.io/v1alpha1
kind: GatewayClass
metadata:
  name: traefik
spec:
  controller: traefik.io/gateway-controller
......
➜ kubectl get gateway -n kube-system
NAME              CLASS     AGE
traefik-gateway   traefik   5m55s
➜ kubectl get gateway -n kube-system traefik-gateway -o yaml
apiVersion: networking.x-k8s.io/v1alpha1
kind: Gateway
metadata:
  name: traefik-gateway
  namespace: kube-system
spec:
  gatewayClassName: traefik
  listeners:
  - port: 8000
    protocol: HTTP
    routes:
      group: networking.x-k8s.io
      kind: HTTPRoute
      namespaces:
        from: Same
      selector:
        matchLabels:
          app: traefik
......

You can see that the Gateway instance created by default refers to the Gateway class of traefik. The listeners part defines the listener entry associated with the Gateway. The listener definition logical endpoint is bound to the Gateway address. At least one listener needs to be specified. The following HTTP route defines the routing rules, Namespaces indicates the namespaces in which routes should be selected for the Gateway. By default, this is limited to the namespace of the Gateway. The Selector specifies a set of route labels. If this Selector is defined, only the objects associated with the Gateway are matched by the route Selector, and an empty Selector matches all objects, Here we will match the object with app: traefik tag.

In order to handle the routing rules in other namespaces, we can use the namespaces From is changed to All, but it has not taken effect after testing?

Now let's install a simple whoami service for testing, and directly use the following resource list to deploy the corresponding service:

# 01-whoami.yaml
---
kind: Deployment
apiVersion: apps/v1
metadata:
  name: whoami
  namespace: kube-system
spec:
  replicas: 2
  selector:
    matchLabels:
      app: whoami
  template:
    metadata:
      labels:
        app: whoami
    spec:
      containers:
        - name: whoami
          image: containous/whoami
          ports:
            - containerPort: 80
              name: http
---
apiVersion: v1
kind: Service
metadata:
  name: whoami
  namespace: kube-system
spec:
  ports:
    - protocol: TCP
      port: 80
      targetPort: http
  selector:
    app: whoami

After the test service deployment is completed, we can use the Gateway API to configure the traffic.

Deploy a simple Host

We will simply deploy an HTTP route object in our previous way.

# 02-whoami-httproute.yaml
apiVersion: networking.x-k8s.io/v1alpha1
kind: HTTPRoute
metadata:
  name: http-app-1
  namespace: kube-system
  labels:
    app: traefik
spec:
  hostnames:
    - "whoami"
  rules:
    - matches:
        - path:
            type: Exact
            value: /
      forwardTo:
        - serviceName: whoami
          port: 80
          weight: 1

The HTTP route resource above will capture the request to the whoami host name and forward it to the whoami service deployed above. If you request this host name now, you will see the typical whoami output:

➜ kubectl apply -f 02-whoami-httproute.yaml
➜ kubectl get httproute -n kube-system
NAME         HOSTNAMES    AGE
http-app-1   ["whoami"]   25s
# Use whoami as the host name for access testing
➜ curl -H "Host: whoami" http://192.168.31.108
Hostname: whoami-6b465b89d6-lcg4k
IP: 127.0.0.1
IP: ::1
IP: 10.244.1.87
IP: fe80::cccc:6aff:fef8:eca9
RemoteAddr: 10.244.1.85:60384
GET / HTTP/1.1
Host: whoami
User-Agent: curl/7.64.1
Accept: */*
Accept-Encoding: gzip
X-Forwarded-For: 192.168.31.9
X-Forwarded-Host: whoami
X-Forwarded-Port: 80
X-Forwarded-Proto: http
X-Forwarded-Server: traefik-84d4cccf9c-2pl5r
X-Real-Ip: 192.168.31.9

In addition, it should be noted that the app: traefik tag needs to be defined in the above HTTPRoute object, otherwise the created Gateway instance cannot be associated with the.

Host with path

The above example can easily restrict traffic to only route on a given sub path.

# 03-whoami-httproute-paths.yaml
---
apiVersion: networking.x-k8s.io/v1alpha1
kind: HTTPRoute
metadata:
  name: http-app-1
  namespace: kube-system
  labels:
    app: traefik
spec:
  hostnames:
    - whoami
  rules:
    - forwardTo:
        - port: 80
          serviceName: whoami
          weight: 1
      matches:
        - path:
            type: Exact  # Path matching / foo
            value: /foo

Create the above modified HTTPRoute, and you will find that the previous request now returns 404 error, while the request / foo path suffix returns success.

➜ curl -H "Host: whoami" http://192.168.31.108
404 page not found
➜ curl -H "Host: whoami" http://192.168.31.108/foo
Hostname: whoami-6b465b89d6-p5vwz
IP: 127.0.0.1
IP: ::1
IP: 10.244.2.154
IP: fe80::7045:53ff:fef9:fadc
RemoteAddr: 10.244.1.85:51686
GET /foo HTTP/1.1
Host: whoami
User-Agent: curl/7.64.1
Accept: */*
Accept-Encoding: gzip
X-Forwarded-For: 192.168.31.9
X-Forwarded-Host: whoami
X-Forwarded-Port: 80
X-Forwarded-Proto: http
X-Forwarded-Server: traefik-84d4cccf9c-2pl5r
X-Real-Ip: 192.168.31.9

More information about which parts of the request can be matched can be found in the official Gateway APIs document( https://gateway-api.sigs.k8s.io/v1alpha1/api-types/httproute/#rules )Found in.

Canary release

Another function that the Gateway APIs specification can support is Canary publishing. If you want to run two different services (or two versions of the same service) on one endpoint and route some requests to each endpoint, you can modify your HTTPRoute.

First, we need to run the second service. Here, we quickly generate an instance of Nginx for testing.

# 03-nginx.yaml
kind: Deployment
apiVersion: apps/v1
metadata:
  name: nginx
  namespace: kube-system
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
        - name: nginx
          image: nginx
          ports:
            - containerPort: 80
              name: http
---
apiVersion: v1
kind: Service
metadata:
  name: nginx
  namespace: kube-system
spec:
  ports:
    - protocol: TCP
      port: 80
      targetPort: http
  selector:
    app: nginx

Next, we modify the previous HTTPRoute resource object. There is a weight option, which can assign different weights to the two services, as shown below:

# 04-whoami-nginx-canary.yaml
---
apiVersion: networking.x-k8s.io/v1alpha1
kind: HTTPRoute
metadata:
  labels:
    app: traefik
  name: http-app-1
  namespace: kube-system
spec:
  hostnames:
    - whoami
  rules:
    - forwardTo:
        - port: 80
          serviceName: whoami
          weight: 3  # 3 / 4 requests to whoami
        - port: 80
          serviceName: nginx
          weight: 1  # 1 / 4 requests to whoami

After creating the above HTTP route, we can now access whoami service again. Normally, we can see that about 25% of requests will see the response of Nginx instead of the response of whoami.

Here we use traifik to test the use of Kubernetes Gateway APIs. At present, the implementation of Gateway APIs by Traefik is based on the v1alpha1 specification. At present, the latest specification is v1alpha2, so there may be some differences with the latest specification.

Added by BLeez on Sat, 12 Feb 2022 02:30:17 +0200

Programming VIP