Version: v26.09

env-check Configuration File (config.json) Analysis of Check Item Meanings and Impacts ​

env-check is distributed to each node via the run command to execute run-local, performing the following 8 checks on each node (run.go:167):

kernel, port, disk, clock, fileQuery, programCheck, route, iptables

log_file ​

AttributeValue
Current configuration"./envCheck.log"
PurposeLog file path for the env-check tool itself

Meaning: All log output from the check process is written to this file, including the start, pass, warning, and failure information for each check item.

Impact: Only affects the env-check tool's own logging, unrelated to BKE initialization.


output_format ​

AttributeValue
Current configuration"text"
Available valuestext, json
PurposeOutput format of the check result report

Meaning: Controls the output format of the check results. text is suitable for command-line viewing, json is suitable for programmatic parsing.

Impact: Only affects the report display format, unrelated to BKE initialization.


paths (File Conflict Check — fileQuery) ​

Check Mechanism ​

Implemented via FileQuery.Execute() in pkg/query/query.go: for each path, it expands environment variables (such as $HOME) and wildcards (such as kube*), then checks whether the file/directory exists. If it exists, it is marked as a "conflict".

Check result determination (run.go:360-367):

  • TotalExists == 0 → pass (all residual paths do not exist)
  • TotalExists > 0 → fail (residual files exist)

14 Paths in Current Configuration ​

No.PathMeaningImpact of Residual on Bootstrap Node InitializationImpact of Residual on Cluster CreationWhether Initialization Handles AutomaticallySeverity
1$HOME/.kubeKubernetes kubeconfig directorybkeadm overwrites ~/.kube/config after starting k3s. If k3s skips startup (residual k3s data causes isKubernetesAvailable to return true), the residual kubeconfig points to the old cluster, and subsequent kubectl/CRD deployment operations hit the wrong clustercapbke overwrites ~/.kube/config on Master nodes; Worker nodes do not generate ~/.kube/config, residual is not overwritten, only affects manual kubectlBootstrap node: conditional overwrite; Master: overwrite; Worker: not touched🟡 Medium
2/etc/kubernetesKubernetes core configuration directory (pki/, manifests/, *.conf)bkeadm does not directly operate on this directory (managed inside the k3s container). If k3s skips startup, residual configuration may cause k3s in-container processes to read old configurationcapbke overwrites most files (pki certificates loaded from Secrets and overwritten, kubeconfig regenerated, manifests overwritten with O_TRUNC), but there may be residual files in the directory that are not overwrittenPartially overwritten, not fully cleaned🟡 Medium
3/usr/bin/kube*Wildcard matching kubectl, kubelet, kubeadm and other binariesbkeadm copies and overwrites /usr/bin/kubectl from the k3s container. The bootstrap node does not install the kubelet binarycapbke first rm -rf /usr/bin/kubelet then downloads (run.go:576-596), kubectl download overwritesAutomatic overwrite🟢 Low
4/usr/local/bin/kube*Wildcard matching k8s binaries under /usr/local/bin/BKE does not install any files in this path, residual will not be overwritten. If /usr/local/bin has higher priority in PATH, the old version may be executedSame as leftNot touched🟢 Low
5/usr/local/bin/crictlcrictl CLI tool (CRI management tool)BKE installs crictl to /usr/bin/crictl (containerd tar extraction), not to /usr/local/bin/crictl. Residual in this path will not be overwritten by BKE. If there is a PATH priority issue, the old version of crictl may be executedSame as leftNot touched (path does not match)🟡 Medium
6/etc/sysctl.d/k8s.confKubernetes kernel parameter configuration (ip_forward, bridge-nf-call-iptables, etc.)bkeadm does not write this file on the bootstrap node (bkeadm's SetSysctl only operates on /etc/sysctl.conf). Residual kernel parameters remain in effect, and incorrect values may cause network anomaliescapbke completely overwrites this file with O_TRUNC on cluster nodes (init.go:342)Bootstrap node: not touched; Cluster node: overwrite🟡 Medium
7/etc/systemd/system/kubelet.servicekubelet systemd unit filebkeadm does not install kubelet (runs inside the k3s container). If the residual file is loaded by systemd, it may cause the old kubelet process to start, conflicting with the kubelet inside the k3s containercapbke completely overwrites with O_TRUNC (run.go:435)Bootstrap node: not touched; Cluster node: overwrite🟡 Medium
8/etc/systemd/system/kubelet.service.dkubelet systemd drop-in directoryNeither bkeadm nor capbke cleans this directory. Residual old drop-in .conf files will be merged and loaded by systemd, overriding the kubelet.service parameters written by capbke, causing kubelet to start with incorrect configurationSame as leftNot cleaned🔴 High
9/var/lib/etcdStandard kubeadm etcd data directoryBKE uses /var/lib/openFuyao/etcd (defaults.go:49), does not use /var/lib/etcd. Residual data will not be read, but indicates the machine previously had standard k8s installedSame as leftNot touched (BKE does not use this path)🟢 Low
10/var/lib/kubeletkubelet working root directory (config.yaml, plugin data, volume mount points)bkeadm does not directly operate on it (kubelet inside the k3s container uses the in-container directory)capbke overwrites config.yaml, but old mount points, Pod data, and the pki/ subdirectory in the directory are not cleaned. Residual old mount points may cause kubelet to attempt mounting non-existent volumes at startupPartially overwritten (only config.yaml)🔴 High
11/run/containerd/containerd.sockcontainerd CRI Unix socketbkeadm installs containerd and starts it, containerd creates this socket. If the old containerd is still running, the socket already exists, and the new containerd may fail to startSame as leftCreated/overwritten when containerd starts🟡 Medium
12/usr/lib/systemd/system/kubelet.service.dkubelet systemd drop-in directory installed by package managerNeither bkeadm nor capbke cleans this directory. Residual old drop-ins will override kubelet.service parametersSame as leftNot cleaned🟡 Medium
13/var/run/containerd/containerd.sockSame as /run/containerd/containerd.sock (/var/run is usually a symlink to /run)Same as item 11Same as item 11Same as above🟡 Medium
14/var/run/docker.sockDocker daemon Unix socketIf the bootstrap node uses containerd mode (default), residual docker.sock indicates the old docker is still running. bkeadm does not check this socketcapbke pre-check detects the container runtime type (check.go:415-430), if docker.sock exists but containerd is configured, pre-check failsNot touched🔴 High

clean_force ​

AttributeValue
Current configurationfalse
PurposeControls whether to skip user confirmation and directly delete residual files in fileClean mode

Meaning:

  • false: Interactively asks the user [y/n] before deleting each residual file.
  • true: Skips confirmation and directly deletes all residual files.

Impact: Only affects the behavior in fileClean mode. The current configuration is false, deletion operations require the user to confirm one by one, preventing accidental deletion.


program_list (Program Check — programCheck) ​

Check Mechanism ​

Implemented via ApplicationChecker.Execute() in pkg/program/check.go: for each program, it uses exec.LookPath(name) to detect whether it can be found in PATH, then compares with the should_exist expected value.

Check result determination (run.go:396-403):

  • TotalFailed == 0 → pass (all programs match the expected state)
  • TotalFailed > 0 → fail (some programs do not match the expected state)

4 Programs in Current Configuration ​

No.Program nameshould_existMeaningImpact of Residual on InitializationInitialization Handling BehaviorSeverity
1dockerfalse (should not exist)Docker container runtimeIn containerd mode, capbke pre-check detects runtime type mismatch (check.go:415-430), node initialization fails. bkeadm does not automatically uninstall dockerDoes not automatically uninstall. bkeadm uses the existing docker if it detects docker is installed; capbke fails the pre-check if it detects a mismatch🔴 High
2kubeletfalse (should not exist)Kubernetes node agent (system service)If the old kubelet is running and not managed by systemd, capbke's systemctl stop kubelet fails (only Warning), the old process continues to occupy ports 10250/10248, and the new kubelet fails to start (run.go:172-176). The old kubelet on the bootstrap node may interfere with iptables and /var/lib/kubeletcapbke first systemctl stop then rm -rf /usr/bin/kubelet then downloads (run.go:576-596), but does not kill running processes not managed by systemd🔴 High
3containerdfalse (should not exist)containerd container runtimeThe binary is overwritten by tar extraction, but residual configuration (/etc/containerd/config.toml) and data (/var/lib/containerd) are not fully cleanedbkeadm extracts the containerd tar to overwrite the binary (containerd.go:172)🟡 Medium
4tartrue (should exist)tar archive tooltar is a basic dependency during the BKE installation process: bkeadm extracts the containerd tar (containerd.go:172), extracts CNI plugins (containerd.go:387), and extracts images/source data (repository.go) all depend on tar. If tar does not exist, initialization fails directlyNot installed (assumed to be provided by the system)🔴 High

Program Check Summary ​

ProgramCheck directionSeverityDescription
dockerShould not exist🔴 HighResidual in containerd mode causes capbke pre-check failure.
kubeletShould not exist🔴 HighResidual running process causes new kubelet port conflict and startup failure.
containerdShould not exist🟡 MediumBinary overwritten, configuration/data residual.
tarShould exist🔴 HighMissing causes all extraction operations to fail.

hosts (Host List) ​

Current Configuration ​

IPRoleSSH port
192.168.2.135bootstrap22
192.168.2.221master22
192.168.2.229worker22

Meaning: Defines the list of target hosts that env-check needs to check. The role of each host determines which port checks are performed (the port list corresponding to the role in port_check.ports).

Impact:

  • In dispatch mode, env-check distributes its own binary to each host and executes checks remotely via SSH.
  • role is used for role matching in port checks (port/check.go:130-155): the bootstrap role performs "bootstrap node port checks", master performs "Master node port checks", worker performs "Worker node port checks".
  • The clock_threshold clock check also requires the hosts list (obtains the time of each host via SSH and compares with the local time).

clock_threshold (Clock Synchronization Check — clock) ​

AttributeValue
Current configuration10 (seconds)
PurposeMaximum allowed time difference between hosts

Check Mechanism ​

Implemented via CheckerClock.Execute() in pkg/clock/clock.go: connects to each host via SSH to obtain the remote time, compares it with the local time, and calculates the time difference. If the time difference between any two hosts exceeds clock_threshold seconds, the check fails.

Impact ​

ScenarioImpactSeverity
Time difference between hosts > 10 secondsThe etcd cluster relies on consistent time for leader election and log replication. Excessive time difference will cause etcd election failure or data inconsistency. Kubernetes certificates also have time validity checks, and time deviation may cause certificate validation anomalies🔴 High
bkeadm initializationbkeadm's setTimezone() sets the timezone and NTP server, but only sets the bootstrap node. Time synchronization of cluster nodes is the responsibility of capbke env init🟡 Medium

Note: The runClockCheck() in the current run.go:327-333 only returns the local time and marks it as pass. The actual multi-host clock comparison is performed by CheckerClock.Execute() in dispatch mode.


kernel_check (Kernel Version Check — kernel) ​

AttributeValue
Current configuration{"min_version": "4.19", "operator": ">="}
PurposeChecks whether the kernel version meets the minimum requirement

Check Mechanism ​

Implemented via Checker.Execute() in pkg/kernel/check.go: uses gopsutil/host.Info() to obtain the kernel version, and compares it with min_version according to operator.

Special handling (check.go:97-115): special judgment for the 3.10.0-xxx kernel of CentOS 7 (with old-style build number) — even if the version number >= 4.19, if the 3.10.0 prefix is detected with an old-style build number, it is also judged as not meeting the >= condition.

Impact ​

ScenarioImpactSeverity
Kernel version < 4.19BKE depends on relatively new kernel features such as cgroup v2, eBPF, and overlayfs. Low-version kernels may cause container runtime, CNI plugin, and kubelet function anomalies. The native snapshotter of containerd and the eBPF datapath of Calico both require a newer kernel🔴 High

port_check (Port Occupancy Check — port) ​

Check Mechanism ​

Implemented via Checker.Execute() in pkg/port/check.go: selects the corresponding port list based on the node role, and performs a TCP dial to 127.0.0.1:<port> for each port. If the connection succeeds, the port is occupied. It can also identify the occupying process via gopsutil.

Check result determination (run.go:274-280):

  • Used == 0 → pass (all ports are free)
  • Used > 0 → fail (occupied ports exist)

Current Configuration ​

Bootstrap Ports (5) ​

PortServiceSource codeConsequence of occupancySeverity
36443Local k3s API Serverbkeadm/constants.go:17 DefaultKubernetesPort🔴 bkeadm validatePorts() hard failure, initialization aborted🔴 High
40080Yum repositorycapbke/defaults.go:63 DefaultYumRepoPort🔴 bkeadm validatePorts() hard failure🔴 High
40443Image repositorycapbke/defaults.go:60 DefaultImageRepoPort🔴 bkeadm validatePorts() hard failure🔴 High
38080Chart repositorybkeadm/constants.go:38 DefaultChartRegistryPort🔴 bkeadm validatePorts() hard failure🔴 High
30010k3s NodePort mapping (ingress-nginx), i.e. the Web access port of the openFuyao management plane (https://<bootstrap-node-IP>:30010)bkeadm/k3s.go:322 -p 30010:30010🟡 k3s container creation failure (validatePorts() does not check this port, the error message may be unclear)🟡 Medium

Among them, ports 36443, 40080, 40443, and 38080 are configurable. If specific ports are used in bke init, they need to be replaced with the corresponding ports to check.

Master Ports (13) ​

PortServiceSource codeConsequence of occupancySeverity
6443kube-apiservercapbke/defaults.go:38 DefaultAPIBindPort🔴 apiserver cannot bind, startup fails🔴 High
2379etcd clientcapbke/consts_sca.go:90 EtcdListenClientPort🔴 etcd cannot bind, startup fails🔴 High
2380etcd peercapbke/consts_sca.go:106 EtcdListenPeerPort🔴 etcd cannot form a cluster🔴 High
2381etcd metricscapbke/consts_sca.go:92 EtcdMetricsPort🟡 etcd metrics cannot bind (non-fatal)🟡 Medium
10248kubelet healthzcapbke/consts_sca.go:295 KubeletHealthzPort🔴 kubelet cannot start🔴 High
10249kube-proxy metricsKubernetes default port🟡 kube-proxy metrics cannot bind🟡 Medium
10250kubelet secure portcapbke/consts_sca.go:350 KubeletPort🔴 kubelet cannot start🔴 High
10256kube-proxy healthzKubernetes default port🟡 kube-proxy healthz cannot bind🟡 Medium
10257kube-controller-managercapbke/consts_sca.go:356 KubeControllerManagerPort🔴 KCM cannot start🔴 High
10259kube-schedulercapbke/consts_sca.go:353 KubeSchedulerPort🔴 scheduler cannot start🔴 High
3377bkeagent-launcher /readyzcapbke/cmd/bkeagent-launcher/main.go:279🟡 launcher readiness probe fails🟡 Medium
58080bkeagent healthbkeadm/constants.go:49 DefaultAgentHealthPort🟡 agent health check fails🟡 Medium
1338containerd metricsbkeadm/.../containerd_default.yaml:27 metricsAddress🔴 containerd startup fails, metrics port binding failure causes process exit🔴 High

Port 1338 is bound to 127.0.0.1 (loopback address), configured by the ContainerdConfig CR (metricsAddress: "127.0.0.1:1338"), and capbke reads this CR and renders it into the containerd config.toml on cluster nodes. The containerd on the bootstrap node uses bkeadm's own template ([metrics] address = ''), does not enable metrics, so the bootstrap port list does not include 1338.

Worker Ports (7) ​

PortServiceSource codeConsequence of occupancySeverity
3377bkeagent-launcher /readyzcapbke/cmd/bkeagent-launcher/main.go:279🟡 launcher readiness probe fails🟡 Medium
58080bkeagent healthbkeadm/constants.go:49 DefaultAgentHealthPort🟡 agent health check fails🟡 Medium
10248kubelet healthzcapbke/consts_sca.go:295 KubeletHealthzPort🔴 kubelet cannot start🔴 High
10249kube-proxy metricsKubernetes default port🟡 kube-proxy metrics cannot bind🟡 Medium
10250kubelet secure portcapbke/consts_sca.go:350 KubeletPort🔴 kubelet cannot start🔴 High
10256kube-proxy healthzKubernetes default port🟡 kube-proxy healthz cannot bind🟡 Medium
1338containerd metricsbkeadm/.../containerd_default.yaml:27 metricsAddress🔴 containerd startup fails, metrics port binding failure causes process exit🔴 High

Port Check Summary ​

RoleNumber of configured portsCommunication matrix coverage
Bootstrap5Covers communication matrix bootstrap TCP ports (36443/40080/40443/38080/30010)
Master13Covers communication matrix master core ports + containerd metrics (6443/2379/2380/2381/10248/10249/10250/10256/10257/10259/3377/58080/1338)
Worker7Covers communication matrix worker core ports + containerd metrics (3377/58080/10248/10249/10250/10256/1338)

disk_check (Disk Space Check — disk) ​

AttributeValue
Current configuration{"check_items": [{"path": "/", "min_free_gb": 50}]}
PurposeChecks whether the available disk space of the specified path meets the minimum requirement

Check Mechanism ​

Implemented via Checker.Execute() in pkg/disk/check.go: for each check_items entry, uses gopsutil/disk.Usage() to obtain the disk usage of the specified path, and compares the Free space with MinFreeGB * 1GB.

If the path does not exist, it automatically searches upward for the mount point of the parent directory. If the check_items entry has a roles field configured, only nodes matching the role are checked (the current configuration does not set roles, so all nodes are checked).

Check result determination (run.go:315-322):

  • InsufficientPath == 0 → pass (all paths have sufficient space)
  • InsufficientPath > 0 → fail (paths with insufficient space exist)

Current Configuration Analysis ​

PathMinimum free spaceMeaning
/50 GBThe root partition requires at least 50 GB of free space

Impact ​

ScenarioImpactSeverity
Root partition free < 50 GBbkeadm validateDiskSpace() checks the disk space of the working directory: if there is existing image data, it requires ≥3 GB, otherwise requires ≥20 GB (initialize.go:217-231). env-check's 50 GB requirement is stricter, reserving sufficient space for BKE installation (image repository, source repository, k3s data, containerd images, etc.)🔴 High

dispatch (Distribution Configuration) ​

AttributeValue
timeout600 (seconds, i.e. 10 minutes)
poll_interval15 (seconds)
work_dir/tmp/envcheck
concurrent_limit10

Meaning of Each Field ​

FieldMeaning
timeoutTimeout for distribution execution. env-check distributes its own binary to each remote node via SCP, executes checks on each node, and collects results. The 600-second timeout covers the check time of all nodes.
poll_intervalInterval for polling remote node check results. Checks whether each node has completed every 15 seconds.
work_dirWorking directory on the remote node. The env-check binary and check result file (result.json) are stored in this directory.
concurrent_limitUpper limit on the number of nodes checked concurrently. At most 10 nodes are checked simultaneously.

Impact ​

ScenarioImpact
timeout too shortNode check not completed but timed out, result marked as failed. Actual checks (8 items) usually complete within 1-2 minutes, 600 seconds is usually sufficient.
work_dir not writableThe /tmp/envcheck directory on the remote node cannot be created (permission issue or /tmp not writable), check results cannot be saved.
concurrent_limit too smallWhen there are many nodes, the check takes a long time. Currently 3 nodes, 10 concurrent is completely sufficient.

Note: The dispatch configuration only affects the distribution behavior of the env-check tool, and is unrelated to BKE initialization.


route and iptables Checks ​

These two checks are executed by default in run-local (run.go:167), but are not explicitly configured in config.json — they use fixed logic and do not depend on configuration parameters.

route (Default Route Check) ​

Check Mechanism (pkg/route/route.go): executes the route -n command to parse the routing table and checks whether a default route exists. If there is no default route, the node's network configuration is abnormal, which may cause communication failure between BKE components.

Impact: Nodes without a default route cannot access the external network (pulling images, downloading binaries, etc.), and BKE initialization fails.

iptables (FORWARD Chain Policy Check) ​

Check Mechanism (pkg/iptables/iptables.go): executes the iptables -L FORWARD -n command to check whether the default policy of the FORWARD chain is ACCEPT.

Impact: If the FORWARD chain policy is DROP, cross-node communication between Kubernetes Pods will be dropped by iptables, causing network unreachability. Calico CNI relies on the FORWARD chain to forward Pod traffic. bkeadm's prepareEnvironment() installs iptables and switches to legacy mode, but does not set the FORWARD policy — this is the responsibility of capbke env init (sets iptables rules in init.go).