Frontend Declarative Upgrade Guide ​

Frontend Declarative Upgrade Introduction ​

Application Scenarios ​

Declarative upgrade is a cluster version upgrade method provided by openFuyao based on declarative APIs. Through frontend upgrades, users only need to specify the target version via the Web interface, and the system will automatically complete the entire process including upgrade path validation, upgrade resource preparation, orderly component upgrades, and status tracking.

Applicable to the following scenarios:

• Upgrading the openFuyao version of a cluster from v26.06-rc.2 and subsequent versions to a higher version.
• Cluster version upgrades in both online and offline environments.

Capability Scope ​

• Declarative upgrade trigger: Users select the target version via the interface, and the system automatically issues upgrade instructions.
• Automatic upgrade path validation: Based on UpgradePath rules, automatically validates whether the upgrade path from the current version to the target version is valid.
• Automatic multi-hop upgrade execution: When no direct upgrade path exists, the system automatically executes upgrades hop by hop through intermediate versions.
• DAG (Directed Acyclic Graph) driven parallel upgrade: Upgrade components build a DAG topology based on dependency relationships, supporting parallel upgrades of components in the same batch.
• Both online and offline scenarios are supported.
• Both management clusters and business clusters can be upgraded. Business cluster upgrades must be performed on the cluster managing their lifecycle (management cluster or bootstrap node).

Highlight Features ​

• Declarative trigger: Only need to declare the target version (modify the desiredVersion field of ClusterVersion CR (Custom Resource)), without worrying about upgrade execution details.
• DAG parallel upgrade: Compared to the legacy serial PhaseFlow upgrade, declarative upgrade supports components executing in parallel based on dependency topology, reducing overall upgrade time.
• Automatic multi-hop execution: During cross-version upgrades, the system automatically calculates intermediate hop versions and executes hop by hop, requiring no manual intervention from users.

Basic Concepts ​

The trigger and execution flow of declarative upgrade is as follows:

Flow step descriptions:

1. User Selects Target Version: The user selects the target version to upgrade to on the interface.
2. Frontend Sends Upgrade Request: The frontend calls the upgrade API and sends the upgrade request.
3. Update ClusterVersion CR: Update the desiredVersion field of ClusterVersion CR to the target version.
4. Validate Upgrade Path: ClusterVersionReconciler queries the UpgradePath CR, validating whether the upgrade path from the current version to the target version is valid.
5. Write Upgrade Annotation: Write the upgrade-ready annotation on BKECluster.
6. Detect Upgrade Annotation: BKEClusterReconciler detects the upgrade-ready annotation on BKECluster and enters the declarative upgrade flow.
7. Build Upgrade DAG: Build the upgrade DAG topology based on the dependency relationships of ReleaseImage and ComponentVersion.
8. Execute in Parallel by Topology Batches: Execute component upgrades in parallel by DAG topology batches; inline components (etcd, kubernetes, etc.) are executed via PhaseRunner, manifest components (provider, coredns, etc.) are executed via ClusterApplier.
9. Determine Upgrade Success: Check whether all components have been upgraded successfully.
10. Upgrade Complete: After successful upgrade, update status and clear upgrade annotations.
11. Record Failed Components: When upgrade fails, record the failed component information.

Usage Restrictions ​

Notice:
For upgrading from v26.03 to v26.06, refer to Upgrade Guide. The following guide applies to upgrading from v26.06-rc.2 and subsequent versions.

Online Scenario Upgrade Operations ​

Prerequisites ​

• The management cluster's openFuyao has been deployed and is running normally.
• The management cluster can access the remote version repository (in online mode, the system automatically fetches version information from the remote repository).
• The cluster to be upgraded has a status of Healthy.

Step 1: Check Cluster Status ​

1. Log in to the openFuyao management plane.

   Note:
   

   • The openFuyao management plane login URL for the bootstrap node is "https://bootstrap node IP address:30010".
   • The openFuyao management plane login URL for the management cluster is "https://management cluster node IP address:31616".
   • For first-time login, the default username is admin, and the default password is test@1234.

2. Select "Cluster Lifecycle Management" in the left navigation bar, find the business cluster to be upgraded, and confirm that the cluster status shows "Healthy".

If the cluster status is not "Healthy", troubleshoot and fix cluster issues before performing the upgrade.

Step 2: Query Upgradable Versions ​

In online mode, the system automatically fetches version information from the remote version repository. In the openFuyao management plane:

1. Select "Cluster Lifecycle Management" and find the business cluster to be upgraded.
2. Enter the cluster's "Upgrade" sub-interface.
3. The system automatically displays the list of target versions that the current cluster can upgrade to (calculated based on UpgradePath rules).

Note:
The upgradable version list is determined by the upgrade path rules in the UpgradePath CR, and only displays versions that the current version can reach through valid paths. If the current version has no reachable upgrade paths, the list will be empty.

Step 3: Select Target Version and Trigger Upgrade ​

1. Select the target version to upgrade to from the upgradable version list.
2. Click "Confirm" to issue the upgrade operation.

The system will automatically execute the following operations:

• Update ClusterVersion CR, setting desiredVersion to the target version.
• ClusterVersionReconciler validates the upgrade path validity.
• If the path is valid, write the upgrade annotation on BKECluster, triggering the declarative upgrade flow.
• BKEClusterReconciler builds the upgrade DAG and executes component upgrades by topology batches.

Note:
1. The actual execution order is determined by the DAG topology; components within the same batch execute in parallel.
2. In multi-hop upgrade scenarios, each hop executes all the above phases before proceeding to the next hop.
3. After provider self-upgrade completes, a new Pod will take over the subsequent upgrade process, and the old Pod will automatically exit.

Step 4: Confirm Upgrade Completion ​

After upgrade completion:

• The cluster status returns to "Healthy".
• The cluster's openFuyao version is updated to the target version; you can view the current version number in the cluster details interface to confirm it matches the target version selected during upgrade.

Offline Scenario Upgrade Operations ​

Prerequisites ​

• The management cluster's openFuyao has been deployed and is running normally.
• The management cluster is in offline mode (cannot access the remote version repository).
• The cluster to be upgraded has a status of Healthy.
• The version configuration file and version offline package have been prepared.

Step 1: Check Cluster Status ​

1. Log in to the openFuyao management plane.
2. Select "Cluster Lifecycle Management" in the left navigation bar, find the business cluster to be upgraded, and confirm that the cluster status shows "Healthy".

Step 2: Upload Version Configuration File ​

1. Log in to the openFuyao management plane of the bootstrap node.

   Note:
   

   • The openFuyao management plane login URL for the bootstrap node is "https://bootstrap node IP address:30010".
   • For first-time login, the default username is admin, and the default password is test@1234.

2. Select "Cluster Lifecycle Management > Patch Upload" in the left navigation bar, and directly upload the version configuration file content.

Note:
In offline scenarios, the version configuration file must be uploaded manually. The system will store the configuration file in the cluster ConfigMap for use during upgrade.

Step 3: Upload Offline Package to Bootstrap Node ​

1. Place the full offline package file in the /etc/openFuyao directory on the bootstrap node.

   Note:
   Only offline packages prepared using the OCI strategy can be uploaded through the openFuyao management plane. The offline package file must be in .tar.gz format.

2. Log in to the openFuyao management plane, select "Cluster Lifecycle Management" in the left navigation bar, find the business cluster to be upgraded, enter the cluster's "Upgrade" sub-interface, select "Upgrade Preparation", enter the offline package file information (path and name), and click "Confirm".

   The system will automatically execute the following operations:

   • Validate the offline package path and format.
   • Extract the offline package (tar.gz).
   • Use the bke registry patch command to synchronize images to the bootstrap node image source.
   • Copy binary files (containerd, kubelet, kubectl) to the /bke/mount/source_registry/files directory.

Note:
1. The "Upgrade Preparation" function on the openFuyao management plane can automatically complete image synchronization and binary file copying.
2. If not using the "Upgrade Preparation" function, you can log in to the bootstrap node backend to manually execute image synchronization and file copying operations.

Step 4: Trigger Upgrade ​

The operation is the same as Online Scenario - Step 3: Select Target Version and Trigger Upgrade. Log in to the openFuyao management plane, select "Cluster Lifecycle Management" in the left navigation bar, find the business cluster to be upgraded, enter the cluster's "Upgrade" sub-interface, select the target version and click "Confirm" to trigger the upgrade.

Step 5: Confirm Upgrade Completion ​

Same as Online Scenario - Step 4: Confirm Upgrade Completion.

Upgrade Failure Handling ​

View Failure Information ​

When upgrade fails:

• Log in to the openFuyao management plane, select "Cluster Lifecycle Management" in the left navigation bar, find the corresponding business cluster, and the cluster status will show "UpgradeFailed".
• In the cluster details, you can view the failed upgrade phase and error information.

Common Failure Causes ​

FAQ ​

• What is the difference between declarative upgrade and legacy upgrade?

  The legacy upgrade (v26.03 and earlier versions) uses the serial PhaseFlow method, where all upgrade phases execute in a fixed sequential order. Declarative upgrade uses the DAG-driven parallel method, where upgrade components execute in parallel by dependency topology batches, reducing overall upgrade time. Declarative upgrade also supports automatic multi-hop execution.

• Why is the upgrade version list empty?

  Possible causes: 1) The current version is already the latest version with no higher versions to upgrade to; 2) The UpgradePath rules have no reachable upgrade paths for the current version; 3) In online mode, the remote version repository is unreachable.

• Why does offline patch upload fail with the prompt "online mode detected"?

  The system detected that the management cluster is in online mode; manual patch file upload via the interface is not supported in online mode. Please check the cluster deployment mode configuration.

• What is multi-hop upgrade?

  When no direct upgrade path exists from the current version to the target version (such as v26.06-rc.2 to v26.06), the system automatically calculates the intermediate hop version path (such as v26.06-rc.2 -> v26.06-rc.3 -> v26.06) and executes upgrades hop by hop. After each hop completes, it automatically proceeds to the next hop until reaching the final target version.