Node Label Affinity Deployment Guide ​

Background ​

When using the BKE installation and deployment tool for cluster construction, users can implement:

1. Node Label Capability: Add custom labels to nodes in the cluster to describe node characteristics (such as hardware specifications, business purposes, etc.), supporting multi-label configuration.
2. Addon Affinity Deployment: During Addon component deployment, perform affinity scheduling based on node labels to ensure specific Addons are only deployed on nodes that meet conditions.

Prerequisites ​

• Bootstrap node initialization is complete, and the bootstrap cluster is running normally.
• Cluster configuration files are prepared.

Note:

The bke-cluster.yaml and bke-node.yaml referenced below are files on the bootstrap node for configuring cluster installation information, located in the /bke/cluster/ directory. These files are used to define cluster configuration, Addon configuration, node information, and other content.

Restrictions ​

• Node label key and value must comply with Kubernetes label naming conventions.

Procedure ​

1. Add labels to nodes.

   In the spec section of the bke-node.yaml configuration file, add the labels field for target nodes:

   yaml

   apiVersion: bke.bocloud.com/v1beta1
   kind: BKENode
   metadata:
     name: master-1
     namespace: bke-cluster
     labels:
       cluster.x-k8s.io/cluster-name: bke-cluster
   spec:
     hostname: master-1
     ip: 192.168.100.226
     password: '******'
     port: "22"
     role:
     - master/node
     - etcd
     username: root
     labels:
       - key: "machine-48"
         value: "true"
       - key: "gpu"
         value: "nvidia"

   Note:

   • Multiple labels can be configured for each node.
   • Labels will be automatically applied to corresponding nodes when the cluster is created.

2. Add nodeSelector support in Addon template.

   In the Addon deployment YAML file under the bke-manifests/kubernetes/<addon-name>/<version>/ directory, add nodeSelector template variable to Pod specification:

   yaml

   # Example: bke-manifests/redis/latest/deployment.yaml
   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: redis
   spec:
     template:
       spec:
         nodeSelector: {{ .nodeSelector }}  # Add this line to support node selection
         containers:
         - name: redis
           image: {{ .imageRepo }}/redis:latest

   Note:

   • {{ .nodeSelector }} will be replaced with the JSON object configured in bke-cluster.yaml during deployment.
   • Only Addon templates that require affinity scheduling need to be modified.

3. Configure Addon node affinity parameters.

   In the addons section of bke-cluster.yaml, add nodeSelector parameter for Addons requiring affinity scheduling:

   yaml

   spec:
     clusterConfig:
       addons:
       - name: redis
         param:
           nodeSelector: '{"machine-48":"true"}'
         version: latest

   Note:

   • The value of nodeSelector is a JSON format string and must be wrapped in single quotes.
   • Ensure target nodes have corresponding labels configured, otherwise Addon cannot be scheduled.

4. Create cluster.

   Execute cluster creation command:

   bash

   bke cluster create -f /path/to/bke-cluster.yaml -n /path/to/bke-node.yaml

5. Verify affinity deployment.

   After cluster creation is complete, verify whether Addon is deployed on specified nodes:

   bash

   kubectl get pod -o wide -A | grep redis

   Confirm whether the node where Pod is located contains configured labels:

   bash

   kubectl get node <node-name> --show-labels

Configuration Example ​

The following is a complete configuration snippet including node labels:

bke-cluster.yaml:

spec:
  clusterConfig:
    addons:
    - name: kubeproxy
      param:
        clusterNetworkMode: calico
      version: v1.33.1
    - name: calico
      param:
        calicoMode: vxlan
      version: v3.27.3
    - name: coredns
      version: v1.10.1
    - name: redis
      param:
        nodeSelector: '{"machine-48":"true"}'
      version: latest

bke-node.yaml:

apiVersion: bke.bocloud.com/v1beta1
kind: BKENode
metadata:
  name: master-1
  namespace: bke-cluster
  labels:
    cluster.x-k8s.io/cluster-name: bke-cluster
spec:
  hostname: master-1
  ip: 192.168.100.***
  password: '******'
  port: "22"
  role:
  - master/node
  - etcd
  username: root
  labels:
    - key: "machine-48"
      value: "true"