Preprocessing and Postprocessing Guide ​

This document aims to introduce preprocessing and postprocessing during cluster installation. The former is executed during cluster node environment initialization, while the latter is executed after issuing all addon installation tasks.

Background ​

For clusters deployed using openFuyao, preprocessing and postprocessing operations need to be executed on cluster nodes, and can implement user-defined operation content.

Prerequisites ​

None.

Restrictions ​

• ConfigMap data for preprocessing and postprocessing must be placed under the user-system namespace.
• Both preprocessing and postprocessing require a global configuration to specify operations to execute.

Usage Scenarios ​

Divided into preprocessing and postprocessing, clusters can contain either one or both.

Preprocessing (PreProcess) ​

This section introduces the global configuration usage for preprocessing, applicable to scenarios where all nodes uniformly execute scripts.

Function and Execution Timing ​

Preprocessing is executed during the EnsureNodesEnv stage of cluster deployment flow. The controller checks for the existence of global configuration, and if it exists, issues the built-in command Preprocess to all matching nodes to execute scripts.

Global Configuration ConfigMap ​

Global configuration only needs to create one ConfigMap, which must meet the following requirements:

• Namespace is user-system.
• Name is preprocess-all-config.
• data name is config.json, content is the global configuration for preprocessing operations.

apiVersion: v1
kind: ConfigMap
metadata:
  name: preprocess-all-config
  namespace: user-system
data:
  config.json: |
    {
      "scripts": [
        {
          "scriptName": "init-os.sh",
          "order": 10,
          "params": {
            "HTTP_REPO": "http://repo.local"
          }
        }
      ]
    }

Field descriptions:

• scripts: Script list (required).
• scriptName: Script ConfigMap name (must match).
• order: Execution order (from small to large).
• params: Script parameters (optional).

Preprocessing Operation ConfigMap ​

Each preprocessing operation must create a separate ConfigMap, which must meet the following requirements:

• Namespace is user-system.
• Name matches the scriptName field in global configuration.
• Label is bke.preprocess.script: "true".
• data must have a key with the same name, content is script text.

Below is an example:

apiVersion: v1
kind: ConfigMap
metadata:
  name: init-os.sh
  namespace: user-system
  labels:
    bke.preprocess.script: "true"
data:
  init-os.sh: |
    #!/bin/sh
    echo "NODE_IP=${NODE_IP}"
    echo "HTTP_REPO=${HTTP_REPO}"

Parameter Rendering and Validation Rules ​

Scripts support ${PARAM_NAME} variable substitution.

• Built-in parameters: NODE_IP, can be automatically obtained from the node's IP address list.
• Custom parameters: Come from params in global configuration.

Parameter name rules:

• Must match: ^[a-zA-Z_][a-zA-Z0-9_]*$.

Parameter value rules (whitelist):

• Only allows: a-z A-Z 0-9 - _ / . space : # =.
• Maximum length for single value: 4096 bytes.

If the custom parameter passed in is illegal, the script will be skipped and will not cause overall failure.

Execution and Writing to Disk ​

The reconciler issues the Preprocess built-in command custom resource to APIServer. After bkeagent monitors the command, it executes. When nodeIP is not explicitly passed in, bkeagent automatically obtains the current node IP address (the IP address bound to the first non-localhost network interface). Finally, the rendered content is written to the /etc/bkeagent/scripts directory.

Usage Example (Complete Process) ​

1. Prepare preprocessing yaml file.

   yaml

   apiVersion: v1
   kind: ConfigMap
   metadata:
     name: init-os.sh
     namespace: user-system
     labels:
       bke.preprocess.script: "true"
   data:
     init-os.sh: |
       #!/bin/sh
       echo "NODE_IP=${NODE_IP}"
       echo "HTTP_REPO=${HTTP_REPO}"

2. Prepare global configuration yaml file.

   yaml

   apiVersion: v1
   kind: ConfigMap
   metadata:
     name: preprocess-all-config
     namespace: user-system
   data:
     config.json: |
       {
         "scripts": [
           {
             "scriptName": "init-os.sh",
             "order": 10,
             "params": {
               "HTTP_REPO": "http://repo.local"
             }
           }
         ]
       }

3. Create cluster ConfigMap data.

   Execute the following commands to create namespace and ConfigMap data.

   bash

   # If user-system namespace doesn't exist, need to create it first
   kubectl create namespace user-system
   
   # Assuming step 1 yaml file name is pre_deal.yaml, step 2 file is pre_cfg.yaml
   kubectl apply -f pre_deal.yaml -f pre_cfg.yaml

4. After applying the above two ConfigMaps, all qualified nodes will execute init-os.sh during the EnsureNodesEnv stage.

Postprocessing (postprocess) ​

This section only introduces the global configuration usage for postprocessing, applicable to scenarios where all nodes uniformly execute scripts.

Function and Execution Timing ​

Postprocessing is executed during the EnsureNodesPostProcess stage of cluster deployment flow. The controller checks for the existence of global configuration, and if it exists, issues the built-in command Postprocess to all matching nodes to execute scripts.

Global Configuration ConfigMap ​

Global configuration only needs to create one ConfigMap, which must meet the following requirements:

• Namespace is user-system.
• Name is postprocess-all-config.
• data name is config.json, content is the global configuration for postprocessing operations.

apiVersion: v1
kind: ConfigMap
metadata:
  name: postprocess-all-config
  namespace: user-system
data:
  config.json: |
    {
      "scripts": [
        {
          "scriptName": "post-clean.sh",
          "order": 10,
          "params": {
            "LOG_DIR": "/var/log"
          }
        }
      ]
    }

Global configuration field descriptions:

• scripts: Script list (required).
• scriptName: Script ConfigMap name (must match).
• order: Execution order (from small to large).
• params: Script parameters (optional).

Postprocessing Operation ConfigMap ​

Each postprocessing operation must create a separate ConfigMap, which must meet the following requirements:

• Namespace is user-system.
• Name matches the scriptName field in global configuration.
• Label is bke.postprocess.script: "true".
• data must have a key with the same name, content is script text.

Below is an example:

apiVersion: v1
kind: ConfigMap
metadata:
  name: post-clean.sh
  namespace: user-system
  labels:
    bke.postprocess.script: "true"
data:
  post-clean.sh: |
    #!/bin/sh
    echo "post clean on ${NODE_IP}"
    echo "log dir is ${LOG_DIR}"

Parameter Rendering and Validation Rules ​

Scripts support ${PARAM_NAME} variable substitution.

• Built-in parameters: NODE_IP, can be automatically obtained from the node's IP address list.
• Custom parameters: Come from params in global configuration.

Parameter name rules:

• Must match: ^[a-zA-Z_][a-zA-Z0-9_]*$.

Parameter value rules (whitelist):

• Only allows: a-z A-Z 0-9 - _ / . space : # =.
• Maximum length for single value: 4096 bytes.

If the custom parameter passed in is illegal, the script will be skipped and will not cause overall failure.

Execution and Writing to Disk ​

The reconciler issues the Postprocess built-in command custom resource to APIServer. After bkeagent monitors the command, it executes. When nodeIP is not explicitly passed in, bkeagent automatically obtains the current node IP address (the IP address bound to the first non-localhost network interface). Finally, the rendered content is written to the /etc/bkeagent/scripts directory.

Usage Example (Complete Process) ​

1. Prepare postprocessing yaml file.

   yaml

   apiVersion: v1
   kind: ConfigMap
   metadata:
     name: post-clean.sh
     namespace: user-system
     labels:
       bke.postprocess.script: "true"
   data:
     post-clean.sh: |
       #!/bin/sh
       echo "post clean on ${NODE_IP}"
       echo "log dir is ${LOG_DIR}"

2. Prepare global configuration yaml file.

   yaml

   apiVersion: v1
   kind: ConfigMap
   metadata:
     name: postprocess-all-config
     namespace: user-system
   data:
     config.json: |
       {
         "scripts": [
           {
             "scriptName": "post-clean.sh",
             "order": 10,
             "params": {
               "LOG_DIR": "/var/log"
             }
           }
         ]
       }

3. Create cluster ConfigMap data.

   Execute the following commands to create namespace and ConfigMap data.

   bash

   # If user-system namespace doesn't exist, need to create it first
   kubectl create namespace user-system
   
   # Assuming step 1 yaml file name is post_deal.yaml, step 2 file is post_cfg.yaml
   kubectl apply -f post_deal.yaml -f post_cfg.yaml

4. After applying the above two ConfigMaps, all qualified nodes will execute post-clean.sh during the EnsureNodesPostProcess stage.