Terminology

Full Name	Abbreviation	Description
addon	-	Components that need to be installed when creating a cluster using the openFuyao community BKE installation tool, such as calico.
All-In-One	AIO	Kubernetes and fuyao-system components are all deployed on the same node.
Admission Webhook	-	Kubernetes admission control mechanism that intercepts and modifies (Mutating) or validates (Validating) requests before API objects are persisted.
API Gateway	APIG	A single entry point between clients and APIs. It acts as a reverse proxy that routes client requests to a group of backend APIs.
Application Programming Interface	API	A set of predefined functions that provide application programs for developers to access a group of routines based on software or hardware, without needing to access the source code or understand the internal details of a mechanism.
Ascend Computing Language	ACL	Provides runtime management, single-operator invocation, model inference, media data processing, and other APIs, enabling the use of underlying hardware computing resources for deep learning inference computation, graphics and image preprocessing, and single-operator accelerated computation on the CANN platform.
AscendHub	-	Docker image repository opened in Ascend.
API Server	API server	Kubernetes API server, providing the cluster's REST API interface.
blackbox_exporter	-	One of the official Prometheus exporters. It enables network probing over HTTP, HTTPS, DNS, TCP, and ICMP.
Bootstrap Node	-	The first node created during cluster initialization, used to bootstrap the entire cluster creation process.
BatchTransfer	-	BatchTransfer encapsulates operation requests, specifically responsible for synchronizing Read/Write data between a non-continuous set of data spaces in one Segment and corresponding spaces in another set of Segments.
Mooncake CacheTier	CacheTier	Different tier cache layers in the Tier Backend architecture of Mooncake, used for hierarchically storing KVCache data.
cAdvisor	-	A container monitoring tool developed by Google. It is embedded in Kubernetes as a monitoring component.
Cloud Native Colocation	-	A deployment approach that uses cloud-native methods to deploy online and offline services in the same cluster, improving overall cluster resource utilization by adjusting resource usage for online services during peak and off-peak periods.
Cloud Native Computing Foundation	CNCF	An open-source software foundation.
ConfigMap	-	An API object in Kubernetes used to store non-confidential data in key-value pairs.
Console	-	The frontend web-based control interface.
Container	-	A runtime instance created based on an image. Containers can be started, started, stopped, and removed. Each container is an isolated and secure platform.
Container memory sharing	-	Based on the UB memory pooling mechanism, when the memory usage rate of a bare-metal container scenario node or NUMA reaches a threshold, memory borrowing is triggered, transparently shifting part of the memory pressure to the remote memory pool.
Container memory borrowing	-	The memory pooling component of UBS-Core, supporting the import and export of memory blocks within the UBS Server cluster through memory pooling capability, achieving cross-node and multi-process shared use of memory on bare metal.
Coordinated Universal Time	UTC	A time standard used to unify the time globally.
CronJob	-	Creates Jobs on a repeating schedule.
Custom Resource	CR	Custom resources in Kubernetes.
Custom Resource Definition	CRD	A Kubernetes resource extension mechanism that allows defining custom resources.
Certificate Authority	CA	An authoritative organization responsible for issuing and managing digital certificates.
Certificate Signing Request	CSR	A file containing certificate applicant information, used to apply for a certificate from a CA.
Common Name	CN	A field in X.509 certificates, typically used to identify the name of the certificate holder.
Certificate	CRT	Typically used to store X.509 certificates.
Configuration Map	ConfigMap	A configuration object in Kubernetes, used to store non-sensitive configuration data.
Certificate Revocation List	CRL	A list containing revoked certificates.
Controller Manager	controller-manager	Kubernetes controller manager, running various controllers to maintain cluster state.
Customstatlogger	-	vLLM exposes the StatLoggerBase abstract class, enabling custom metrics and metric reporting methods.
DaemonSet	-	Ensures that all (or some) nodes run a copy of a Pod.
Dashboard	-	A dashboard consists of multiple monitoring components customized by users. Users can monitor various indicators based on their requirements.
Data Parallelism	DP	A full copy of a model is stored on each device. Each device independently processes a part of the dataset, and then summarizes the respective gradients.
Decode	-	Process from the generation of the first token to the end of inference.
Deployment	-	Provides declarative update capabilities for Pods and ReplicaSets.
DeviceClass	-	A cluster-level API object in the Kubernetes DRA framework, used to define an abstract category of a certain type of device resources and its selection semantics, for ResourceClaim to reference when requesting resources.
Device-to-Device Transmission	D2D Transmission	A transmission method where data is directly moved within the same accelerator card or between different accelerator cards without going through the CPU or host memory.
Domain Name System	DNS	A service that maps domain names to IP addresses for easier network access.
Dubbo	-	An open-source, high-performance service framework from Alibaba that delivers transparent RPC remote invocation and service governance.
Dynamic Resource Allocation	DRA	A device resource management mechanism provided by Kubernetes, used to implement device resource request, scheduling, and allocation through structured API objects, and allows device drivers to participate in the resource selection and allocation decision process.
DCGM Exporter	-	Collects GPU runtime and health indicators, including GPU utilization, PCIe transfer rate, temperature, power usage, and other information.
Embedding	EMB	Data vectorization embedding operation.
Extended Key Usage	extKeyUsage	A certificate extension field that specifies the specific uses of a certificate (such as server authentication, client authentication, etc.).
etcd	-	A distributed key-value storage system, used by Kubernetes to store cluster state and configuration data.
EndpointPicker	EPP	In the Kubernetes Gateway API Inference Extension, a component responsible for selecting appropriate backend instances, supporting endpoint selection based on different routing strategies.
Felix	-	Calico's node agent component, running on each node, responsible for configuring network, routing, and policy rules.
Fully Qualified Domain Name	FQDN	Name with both the hostname and domain name in the format of FQDN=Hostname+DomainName.
Gigabyte	GB	A decimal unit of information measurement, commonly used to denote the storage capacity of disks or memory with large capacities.
Gateway API Inference Extension	GIE	Extends inference-related capabilities on top of the Kubernetes Gateway API, used to define and manage routing and traffic policies for inference services.
Helm	-	A package manager in Kubernetes, used to simplify the deployment and management of applications in Kubernetes clusters.
Helm Chart	-	A core concept in Helm, which is a pre-configured package of application resources.
High Availability	HA	The ability of a system or service to run with high reliability and continuous availability, even in the face of hardware failures or other abnormal situations.
High Performance RSA Engine	HPRE	KAE high-performance RSA acceleration engine module.
High Performance ZIP Engine	ZIP	KAE high-performance zlib/Gzlib compression engine module.
Horizontal Pod Autoscaler	HPA	Automatically updates workload resources (such as Deployment or StatefulSet), with the purpose of automatically scaling workloads to meet demand.
Host-to-Device Transmission	H2D Transmission	The process of copying data from CPU/host memory to NPU/GPU and other accelerator device memory.
Hypertext Transfer Protocol Secure	HTTPS	A secure HTTP channel that ensures transmission security through encryption and authentication on top of HTTP.
Indicator	-	A monitoring indicator is an indicator supported by the data collection system (such as Prometheus) for users to monitor. A monitoring indicator can contain multiple monitoring instances.
Ingress	-	An API object that manages external access to services in a cluster, providing load balancing, SSL termination, and name-based virtual hosting.
Instance	-	A monitoring instance is the smallest monitorable object in Kubernetes. Each monitoring instance is uniquely identified by a set of key-value pair labels.
Job	-	Runs one-off tasks in the cluster, focusing on executing one-off tasks rather than maintaining a specified number of running instances. The Job controller creates one or more Pods to run the specified task. When the task is completed, the Job controller deletes the Pods.
Key-Value Cache	KVCache	A common strategy for accelerating large model inference, working by caching the Key (K) and Value (V) matrices generated by the self-attention mechanism during large model inference, avoiding redundant computations and improving inference speed.
kube-apiserver	-	Validates and configures data for API objects, including Pods, Services, ReplicationControllers, etc. The API server provides services for REST operations and provides the frontend for the cluster's shared state, through which all other components interact.
kubectl	-	A command-line tool for communicating with the Kubernetes cluster control plane using the Kubernetes API.
kubelet	-	An important component in the Kubernetes cluster, running on each node, responsible for managing containers on that node. It is the node agent in the Kubernetes system, communicating with controllers on the main control plane to ensure containers run on nodes as expected.
Kube-rbac-proxy	-	A lightweight HTTP proxy service designed for Kubernetes, utilizing the Kubernetes SubjectAccessReview feature to enforce RBAC (role-based access control) authorization. The goal of this project is to restrict communication between Pods, allowing only Pods with valid and RBAC-authorized tokens to access other Pods.
Kubernetes	K8s	A portable, extensible, open-source platform for managing containerized workloads and services, facilitating declarative configuration and automation.
kube-state-metrics	-	Collects status indicators of resource objects generated by the API server, such as Deployment, Node, Pod.
Kunpeng Accelerator Engine	KAE	A hardware acceleration solution provided based on Kunpeng 920 processors.
Key	KEY	Used to store private keys.
Kubernetes Configuration	kubeconfig	Contains cluster access information, authentication information, and context configuration.
Key Usage	keyUsage	A certificate extension field that specifies the uses of the certificate key (such as digital signature, key encryption, etc.).
kube-proxy	-	Kubernetes network proxy, responsible for maintaining network rules and load balancing on nodes.
Large Language Model	LLM	A deep learning model trained on a large amount of text data.
Load Balancer	-	A device or service used to distribute network traffic among multiple servers.
metrics-server	-	One of the core components in the Kubernetes monitoring system, responsible for collecting resource indicators from Kubelet, then aggregating the monitoring data (depending on kube-aggregator), and exposing them through the Metrics API (/apis/metrics.k8s.io/) in the Kubernetes Apiserver. However, metrics-server only stores the latest indicator data (CPU/Memory).
Mind Inference Service	MIS	An LLM inference API service provided by Ascend based on containerized deployment.
Mind Inference Service Operator	MIS-Operator	A component that implements lifecycle management of inference microservice instances.
Mooncake	-	A distributed KVCache storage engine designed for large model inference, improving inference efficiency.
Multi-core	-	Integrating a large number of processing cores on a single chip. Multi-core scenarios specifically refer to nodes with more than 256 CPUs in a cluster.
Mutual TLS	mTLS	A two-way encrypted channel is used between a server and a client.
Mooncake	-	An open-source community that proposes a KVCache-centric LLM service decoupling architecture.
Mooncake Store	-	A high-performance distributed key-value KVCache storage engine designed for LLM inference scenarios.
Mooncake Store Master Service	Master Service	Responsible for managing the logical storage space pool of the entire cluster in Mooncake Store, and handling node join and exit events.
Mooncake Store Client	Mooncake Client	The client of Mooncake Store, responsible for initiating get/put requests called by upper-layer applications and providing actual KVCache storage.
Mooncake Transfer Engine	TE	A high-performance, zero-copy data transfer library designed around the two core abstractions of Segment and BatchTransfer.
Namespace	-	A Kubernetes namespace, which is an isolated resource space within a project on the platform, and also the user's workspace for production. A project can create multiple namespaces, and the total occupied resource quotas cannot exceed the project quota. Namespaces provide finer-grained resource quota division while also limiting container sizes (CPU, memory) within the namespace, effectively improving resource utilization.
Nginx	-	A high-performance HTTP and reverse proxy web server that also provides IMAP/POP3/SMTP services.
Node	-	Depending on cluster configuration, a node can be a virtual machine or a physical machine.
Node Feature Discovery	NFD	Kubernetes node feature discovery function. It detects available hardware features on each node in a Kubernetes cluster and advertises them using node labels, annotations, and node taints.
node_exporter	-	Used to collect and expose host system indicators, such as CPU, disk, memory, network, etc. It can be used with Prometheus or other monitoring tools, and supports various collectors and custom indicators.
Non-Uniform Memory Access	NUMA	NUMA is a memory architecture in modern multi-core and multi-processor systems that optimizes memory access speed by dividing processors and memory into multiple nodes.
NATS	-	NATS is an open-source, lightweight, high-performance distributed messaging system that provides publish/subscribe, request/reply, and queue subscription communication models.
NATS Prometheus Exporter	-	Collects NATS server monitoring endpoint indicators (such as varz, connz, subsz, routez), including connection count, subscription count, message throughput, transfer rate, client latency, and other information, used for monitoring the performance and health status of the messaging system.
OAuth2-Server	-	A server providing the OAuth 2.0 protocol implementation in openFuyao.
OAuth-proxy	-	Provides OAuth2-based authentication and authorization functionality. It helps protect web applications or APIs, ensuring that only authenticated users can access protected content.
Offline Workload	-	Services with relatively low service quality requirements and insensitive to response delay, such as big data analysis, transcoding, AI training, etc.
Online Workload	-	Services with high service quality requirements and sensitive to response delay, such as web services, e-commerce, etc.
Open Authorization 2.0	OAuth2.0	An industry-standard authorization protocol. OAuth 2.0 focuses on simplifying client developer work while providing specific authorization flows for web applications, desktop applications, mobile phones, and living room devices.
Operating System	OS	A built-in program used to coordinate various computer hardware and interact with users. Common ones include Windows, macOS, and open-source Linux.
Organization	O	A field in X.509 certificates used to identify the organization to which the certificate holder belongs.
Persistent Volume	PV	A piece of storage in a cluster, prepared and managed by an administrator.
Persistent Volume Claim	PVC	A Kubernetes storage resource used for file storage.
Pipeline Parallel	PP	A technology that splits a model into multiple parts by layer and computes on different devices.
Pod	-	The smallest deployable computing unit that can be created and managed in Kubernetes.
Prefill	-	Process from the user completing the prompt input to the generation of the first token.
Prefill-Decode Disaggregation	PD	An architecture that schedules the Prefill and Decode phases of large model inference to different hardware clusters for execution, to optimize resource configuration and improve system performance.
Prometheus	-	An open-source system monitoring and alerting toolkit used to collect and process real-time indicators. It periodically pulls monitoring data from target services or agents through the HTTP protocol and stores it in a highly available time series database. Users can query, aggregate, and visualize this data using the PromQL query language, and trigger alerts based on predefined rules.
Prompt	-	Information input by the user to the model, and the model generates output that meets expectations.
Public Key Infrastructure	PKI	A system for managing digital certificates and public-private key pairs.
Public-Key Cryptography Standards #1	PKCS#1	RSA encryption standard that defines the storage format of RSA private keys.
Privacy-Enhanced Mail	PEM	A Base64-encoded certificate and key storage format.
Profile	-	A configuration item in the certificate issuance policy configuration, defining the issuance parameters for a specific type of certificate.
PodGroup	-	A job developer declares a PodGroup, and the scheduler performs resource judgment and reservation in "group" units, ensuring that all Pods of the job can start simultaneously (Gang Scheduling).
Quality of Service	QoS	Kubernetes classifies Pods into three QoS levels (Guaranteed, Burstable, and BestEffort) based on their resource requests and limits, used to determine resource allocation and eviction priority. Workloads can also customize QoS policies to differentiate priorities and resource guarantees across different services or tasks. QoS helps improve the stability of critical services and resource utilization.
RayCluster	-	A basic Ray cluster consisting of 1 head node and 0 to several worker nodes forming an application cluster.
RayJob	-	Used to submit and execute a single job. Each submitted job independently creates a Ray cluster, executes the task once the cluster is ready, and automatically destroys it after task completion, achieving cluster-level isolation.
RayService	-	Deploys Ray Serve, creating an independent Ray cluster during deployment, and supporting service hot updates, high availability, and other capabilities.
Resource Acquisition Is Initialization	RAII	A C++ resource management paradigm that binds the lifecycle of memory, file handles, locks, and other resources to local objects, acquiring resources through construction and automatically releasing them through destruction, reducing resource leaks and improving exception safety.
Remote Direct Memory Access	RDMA	A technology for directly accessing the memory of one computer from another computer's memory. It enables network cards to directly access application memory, supporting zero-copy network communication.
Remote Procedure Call	RPC	A computer communication protocol. This protocol allows a program running on one computer to call a subroutine in another address space (typically a computer on an open network), and the programmer can call it like a local program without additional programming for this interaction (i.e., without worrying about details).
Resource	-	Built-in resources and custom resources in Kubernetes.
Resource Overselling	-	When online services in hybrid deployment have high remaining resources during off-peak periods, resource overselling is the behavior of dynamically allocating these resources to offline jobs.
ResourceClaim	-	A namespace-level API object in the Kubernetes DRA framework, used to represent a specific request for a certain type of device resource, and bound to an actual resource instance on a specific node during scheduling and allocation.
ResourceClaimTemplate	-	A namespace-level API object in the Kubernetes DRA framework, used to define a template specification for generating ResourceClaims.
ResourceSlice	-	A cluster-level API object in the Kubernetes DRA framework, used to represent a set of structured resource instances available for allocation by a certain DRA Driver on a specific node.
Role-based Access Control	RBAC	An access control method that manages access permissions to system resources through user roles. In RBAC, permissions are associated with roles, and users acquire corresponding permissions through their assigned roles.
Secret	-	An object containing a small amount of sensitive information such as passwords, tokens, or keys.
Security Engine	SEC	The KAE hardware security acceleration engine module.
Service	-	A method in Kubernetes for exposing network applications running on one or a group of Pods as network services.
Service Level Object	SLO	A goal that ensures the provided services meet customer expectations.
ServiceMonitor	-	One of the core abstractions of Prometheus Operator for the monitoring system, enabling convenient indicator monitoring through ServiceMonitor.
Silence	-	A basic capability provided by alerting components, matching alerts based on configured silence rules. Once a matching alert is found, it is silenced, i.e., not pushed for notification.
Spring Cloud	-	A complete microservices solution suite based on the Spring Boot framework.
StatefulSet	-	A workload API object used to manage stateful applications.
Subject Alternative Name	SAN	An extension field in certificates used to specify multiple domain names or IP addresses.
scheduler	-	Kubernetes scheduler, responsible for scheduling Pods to appropriate nodes for execution.
Segment	-	Represents a continuous address space that can be remotely read and written.
Tensor Parallel	TP	A technology that splits the model weight matrix into multiple parts and computes on different devices.
Tier Backend	-	A backend system in Mooncake that supports tiered storage, responsible for uniformly managing KVCache data on different storage tiers.
Transport Layer Security	TLS	Used to provide encrypted communication on networks.
Time To First Token	TTFT	The latency from input to the output of the first token in large model inference.
Universally Unique Identifier	UUID	UUID is a software construction standard, composed of a timestamp, clock sequence, and a globally unique node identifier (such as a hash of the hostname).
Video Random Access Memory	VRAM	Dedicated high-speed memory for graphics cards, used to temporarily store rendering data such as textures and frame buffers.
Virtual Central Processing Unit	vCPU	A virtual central processing unit, a processor resource used in virtual environments. It is a part of a physical CPU that can be used independently by a virtual machine. Unlike actual physical CPUs, vCPU divides a physical processor into multiple virtual processor cores through hyper-threading technology, enabling resource sharing and dynamic allocation.
VictoriaMetrics	VM	VictoriaMetrics is a high-performance, low-cost, horizontally scalable open-source time series database and monitoring solution, designed for large-scale indicator data storage and query optimization, compatible with the Prometheus ecosystem.
Visual Language Model	VLM	A deep learning model trained on a large amount of visual-text data.
vLLM	-	An efficient inference engine and framework designed for large language models, capable of optimizing large model inference performance.
Volume	-	An abstract concept in Kubernetes used to provide persistent storage for containers within a Pod.
Widget	-	A monitoring component containing a name and data chart, displayed in card form.
-	xPyD	Architecture that contains x P nodes and y D nodes in the PD architecture.
X.509	X.509	A public key certificate standard developed by the International Telecommunication Union (ITU), defining the format and structure of digital certificates.
Equivalent Class Scheduling	-	Treats Pods with the same resource requests, affinity, and other conditions as "equivalent classes", allowing one scheduling decision to apply to the entire class, greatly reducing scheduling computation overhead for large-scale jobs.
Topology-aware Scheduling	-	Combines node network topology (NVLink, RDMA) and hardware information, prioritizing scheduling Pods that require high-speed communication to the same node or adjacent nodes.

View source on GitCode

Bug

Terminology ​

Terminology