Cluster Lifecycle Management

Feature Overview

Installation and deployment serve as the starting point for PaaS platforms and are a key measure of product capability. The openFuyao installation and deployment tool adopts a Cluster API–based approach to install and deploy the system and uses Kubernetes-style declarative APIs to manage multiple Kubernetes clusters. This simplifies cluster lifecycle management, optimizes cluster installation workflows and management details, and enhances automation during installation. This section describes how to install service clusters and manage the cluster lifecycle after the management cluster is installed.

Applicable Scenarios

You can use the installation and deployment tool to create the openFuyao management plane and clusters to install and deploy the container platform.

Supported Capabilities

This feature enables the installation and deployment of the entire container platform. It enables you to manage the lifecycle of service clusters, including creating, removing, scaling up, scaling down, and upgrading service clusters.

Highlights

Based on a Cluster API–based approach, Kubernetes-style declarative APIs are used to manage multiple Kubernetes clusters, simplifying cluster lifecycle management and enhancing automation during installation.

Implementation Principles

• General process

1. A user downloads the BKE installation tool and initializes the bootstrap node.

2. The user uses the bootstrap node to push an agent to the management cluster node. On the management cluster node, the user deploys a Kubernetes cluster and Cluster API to create the management cluster.

3. On the openFuyao management plane, the user can create, uninstall, and dynamically scale up or down service clusters, and upgrade Kubernetes versions (including dynamic scaling of the management cluster).

   NOTE

   • The bootstrap node is used to set up a management cluster, the management cluster is used to set up service clusters, and the service clusters are used to run customer programs.
   • In the current installation and deployment tool, you can initialize the bootstrap node and set up the management cluster on the bootstrap node using commands. You can use the Cluster Lifecycle Management page of openFuyao to create service clusters in the management cluster.

Figure 1 Installation and deployment process

The BKE component in the figure consists of the bkeadm, bkeagent, and Cluster API. The details are as follows:

bkeadm

It is a BKE installation tool and is used only as a binary file on the bootstrap node. Its main functions include starting the image repository and YAML repository, starting the K3s cluster and installing the Cluster API, and submitting the management cluster configuration.

bkeagent

It is a cloud-native host machine management tool that listens to kube-apiserver instructions and executes the instructions on the host machine. Its main functions include listening to CRD resources and reporting execution results, switching the Kubernetes cluster to be listened to, and providing multiple built-in tasks, such as node environment check, Docker installation, and shell script execution.

Cluster API

It is a controller that distributes and starts the bkeagent on all nodes, creates instructions to direct the bkeagent, summarizes the results of the deployed clusters, and updates the results to bkecluster events.

Figure 2 Logical architecture of the system

After the cluster configuration items are modified, the frontend submits the configuration file to the backend. The backend processes the configuration file, calls the APIs, and submits the configuration file to the corresponding Kubernetes API server to perform cluster management operations.

Related Features

None.

Instances

Code links:

https://gitcode.com/openfuyao/installer-website

https://gitcode.com/openfuyao/installer-service

Installation

The installation generally consists of management cluster installation and service cluster installation. For details about how to install the management cluster, see online installation and offline installation. For details about how to install the service cluster, see the following sections.

Logging In to the openFuyao Management Plane

Enter "https://login IP address of the bootstrap node:web service port for cluster lifecycle management" in the address box of a browser and enter the username and password to log in to the Cluster Lifecycle Management page.

NOTE

• Changing the password is required upon the first login.
• The default web service port of the Cluster Lifecycle Management page is 30010.

Adding a Cluster

Prerequisites

• The bootstrap node has been configured.

• The management cluster has been installed.

Context

You can create a service cluster on the Cluster Lifecycle Management page.

Restrictions

• Ensure that the node IP address, username, and password are correct.

• When creating an HA cluster, ensure that the IP address of the load balancer is correct.

Procedure

1. In the left navigation pane, choose Cluster Lifecycle Management. A cluster list is displayed on the Cluster Lifecycle Management page. Cluster information, including the cluster name, status, and number of nodes, is provided.

2. Click Add Cluster in the upper-right corner of the list. The Add Cluster page is displayed.

   Figure 3 Adding a cluster

   

3. Enter related information on the page.

   Table 1 Page description

   Section Name	Description
   Basic Information	Specifies the name for the cluster.
   Master Information	Specify the information about the master node of the service cluster in this section. Click to add more master nodes. Click in the Operation column to remove the added node.
   Node Information	Specify the information about the node of the service cluster in this section.
   Advanced Settings	By default, the preconfigured image repository address, image repository IP address, and chart repository address are used. For online installation, the default value of Repository is the address of the online repository. For offline installation, the default value of Repository is the address configured on the initialized node.

4. Click Create at the bottom. The Cluster Information dialog box is displayed.

5. After all the cluster information is displayed, click OK.

6. The system returns to the list page. Installing is displayed in the Status column.

7. Check the status later to determine whether the cluster is added.

   NOTE
   If an error message is displayed in Cluster Information, locate and resolve the error.

Removing a Cluster

Prerequisites

A cluster has been created on the Cluster Lifecycle Management page.

Context

You can remove a service cluster on the Cluster Lifecycle Management page.

Restrictions

Currently, only clusters that have been successfully created can be removed. For a cluster that fails to be created, you need to troubleshoot the issue and then manually remove it on each node of the cluster. The procedure for manually removing a cluster is as follows:

1. Download and automatically install bkeadm on each node of the cluster.

   curl -sfL https://openfuyao.obs.cn-north-4.myhuaweicloud.com/openFuyao/bkeadm/releases/download/v25.06/download.sh | bash
   

2. Initialize the configuration.

   bke reset --all --mount

3. Remove the BKE file.

   which bke | xargs rm -rf

Procedure

1. On the Cluster Lifecycle Management page, click in the Operation column and select Delete.

2. In the dialog box that is displayed, select "I understand that the deletion operation is permanent and cannot be undone."

3. Click Delete. The message "Deleting... Please wait." is displayed, and Deleting is displayed in the Status column.

4. A cluster is removed when information about it is cleared from the list.

   NOTE
   The removal process usually takes 10–20 minutes, depending on network conditions.

Upgrading a Cluster

Prerequisites

A cluster has been created on the Cluster Lifecycle Management page.

Context

You can perform an in-place Kubernetes upgrade for a service cluster on the Cluster Lifecycle Management page.

Restrictions

• Only upgrades to a higher Kubernetes version are supported. Version rollbacks are not supported.

• In the current version, Kubernetes 1.28.8 is installed by default and can be upgraded to 1.29.1.

Precautions

• An upgrade is irreversible. Proceed with caution.

• Before upgrading a cluster, ensure that the cluster is healthy. You are advised not to perform any operation on the cluster during an upgrade.

• Only in-place upgrades are supported. During an upgrade, services are unavailable.

Procedure

1. Method 1: On the Cluster Lifecycle Management page, click in the Operation column and select Upgrade.

   Method 2: In the upper-right corner of the Cluster Details page, click Upgrade Cluster.

2. In the Upgrade Cluster dialog box that is displayed, select the target version from the drop-down list.

3. Confirm that all information is correct and click OK. The system automatically returns to the list page, and the Status column displays Upgrading. After the Status column indicates a successful upgrade, the cluster upgrade is complete.

   After the upgrade is complete, you can view the latest Kubernetes version of the cluster in Kubernetes version on the cluster list page.

Scaling Up a Cluster

Prerequisites

A service cluster has been created on the Cluster Lifecycle Management page, and the cluster state is Healthy.

Context

You can scale up a service cluster on the Cluster Lifecycle Management page.

Restrictions

During cluster scale-up, ensure that the nodes used for scale-up are not in use. During cluster scale-up, only worker nodes can be added.

Procedure

1. On the Cluster Lifecycle Management page, click a cluster name. The Node Details page is displayed.

   NOTE
   On the Node Details page, the master node is displayed before worker nodes in the node list.

2. Click Add Node in the upper-right corner of the list.

   NOTE
   All newly added nodes must be worker nodes.

3. In the dialog box that is displayed, enter related information, such as the node name, port number, IP address, username, and password.

   • Click to add more nodes.

   • Click in the Operation column to remove the added node.

4. Then, click Next.

5. The Cluster Information dialog box is displayed. After all information is displayed, click OK. The system returns to the Cluster Lifecycle Management page, and the status of the cluster is Installing.

6. Check the status of the cluster later to determine whether the cluster scale-up is complete.

   • If the scale-up fails, the status of the cluster is Installation failed.
   • If the scale-up is successful, the status of the cluster is Healthy.

   NOTE
   Do not perform any other operations on the cluster until the scale-up succeeds because the cluster is unstable during the scale-up.

Scaling Down a Cluster

Prerequisites

A service cluster has been created on the Cluster Lifecycle Management page, and the cluster state is Healthy.

Context

You can scale down a service cluster on the Cluster Lifecycle Management page.

Restrictions

During cluster scale-down, only worker nodes can be removed. Removed nodes cannot be restored. Proceed with caution.

Procedure

1. On the Cluster Lifecycle Management page, click a cluster name. The Node Details page is displayed.

2. Select a removal mode.

• Removing nodes in batch

1. Click the check boxes next to Node name to select the nodes to be removed.

2. Click Delete above the list.

• Removing a single node

1. Select the node to be removed.

2. Click in the Operation column.

3. In the dialog box that is displayed, select "I understand that the deletion operation cannot be undone".

4. In the dialog box that is displayed, click Delete to remove the selected node.

Status Description

During cluster operations (installation, scale-up, or upgrading), the Status column on the page reflects the following states.

Installation

During installation, the status changes are as follows:

• If the cluster has only one worker node, the states include: Installing, Abnormal, and Healthy.

• If the cluster has multiple worker nodes, the states include: Installing, Scaling up, Abnormal, and Healthy.

Scale-up

During cluster scale-up, the states include: Installing, Scaling up, Abnormal, and Healthy.

Upgrade

During a cluster upgrade, the states include: Upgrading, Abnormal, and Healthy.

Actual Actions Performed During Each State

Table 2 State and related actions

State	Related Actions
Installing	Preparing the node environment, integrating with Cluster API, creating cluster certificates, configuring cluster entry, and initializing and adding the master node.
Scaling up	Adding worker nodes to the cluster.
Upgrading	Upgrading the master and worker nodes.
Abnormal	A cluster health check is performed during installation, scale-up, or an upgrade, and some pods are not ready.
Healthy	The cluster is in a healthy state.