Skip to main content

Platform Overview

Platform Architecture​

image

  • The AI platform provides standard, compliant REST APIs externally.

  • Internally, the AI platform comprehensively optimizes open-source tools in terms of functionality, reliability, security, and supporting tools. It integrates core components from various domains, such as self-developed Kubeflow, and independently develops systems for deep learning, cluster management, resource monitoring, intelligent operations, and user management (LDAP, AD, etc.).

  • The AI platform's task scheduling system supports multiple scheduling strategies, including user priority scheduling, job queuing, parallel task gang scheduling, network topology-aware scheduling, and GPU sharing scheduling.

  • The AI platform's intelligent fault tolerance mechanism effectively handles system failures (downtime, network disconnection, accelerator failure) through task fault tolerance and checkpoint-based resume training, supporting single-node, multi-node, and MPI training jobs.

  • The AI platform's high-speed caching system supports shared high-speed caching and node-local caching mechanisms, allowing users to select the caching method best suited for their training workload based on dataset size.

  • The AI platform integrates with data storage systems such as GPFS, NFS, HDFS, and Ceph.

Platform Positioning​

Driven by customer needs and leveraging independent R&D and continuous technological innovation, the AI platform focuses on industry users in security, finance, energy, automotive, and education sectors. It aims to become a leading, high-performance, scalable, and secure enterprise-grade AI platform. The platform provides unified resource management and multi-strategy scheduling for heterogeneous computing clusters, visual AI modeling, AI training jobs, and full lifecycle management for AI service deployment. Its design objectives are:

  • Rich functionality: Heterogeneous computing resource management, container management, streamlined business workflows, and efficient intelligent monitoring.

  • Excellent framework compatibility: Supports mainstream frameworks such as TensorFlow, PyTorch, MXNet, Caffe, and PaddlePaddle, as well as mainstream model serving frameworks.

  • High-performance deep learning training job scheduling system: A self-developed deep learning scheduling system and distributed training strategy designed based on the characteristics of deep learning training jobs and containers.

  • Rapid deployment and iteration: Supports fast deployment and iteration of application services. Leveraging container technology, it enables rapid service startup and response, while providing online rolling updates and A/B testing capabilities to allow service upgrades or testing without downtime.

  • High availability: Deep learning training fault tolerance technology ensures business continuity and uninterrupted platform services.

  • Security: A comprehensive security system featuring tenant isolation and secure access control.

Platform Philosophy​

  • Open: Builds a vendor-agnostic enterprise-grade AI platform centered on containers.

  • Agile: Containerized and visual design enhances the efficiency of building, using, and maintaining customer AI infrastructure.

  • Secure: Hierarchical architecture design isolates resources at theinfrastructure level and employs secure sandboxes for data security, achieving end-to-end security isolation from business applications to underlying resources.

  • Intelligent: Provides an intelligent experience for core needs such as resource management and job scheduling, based on fine-grained, efficient monitoring and data analysis. Intelligent gateway technology enables efficient service scaling to increase system resource utilization.

Platform Features​

One-stop AI Platform:

  • Supports deep learning task scheduling to improve training and service deployment efficiency. Enables the development, training, and deployment of AI models within a single platform, achieving streamlined, integrated management of models and container images.

  • Open compatibility with multiple computing frameworks: Caffe, TensorFlow, PyTorch, MXNet, PaddlePaddle, etc.

  • Provides a deep learning development environment to simplify network model development. Users can quickly build deep learning environments on demand, define the number and configuration of containers, and elastically scale container compute services. When container capacity is insufficient, users can easily expand their services.

  • Leverages the advantage of abundant GPU cluster computing resources to provide a quick entry point for distributed deep learning training, simplifying the steps for distributed training.

Rich Operations and Monitoring Features:

  • Provides analytical and statistical functions, allowing viewing and analysis by weekly, monthly, or yearly cycles.

  • Comprehensive cluster monitoring and management to track CPU, accelerator, and other resource usage and operational status in real time.

  • One-stop Kubernetes cluster management and maintenance.

  • Real-time detection of component failures within the cluster, triggering alerts and event logs, and automatically completing fault recovery; improves platform availability and reduces operations and maintenance costs.

Platform Overview​

Multiple Data Usage Methods​

The AI platform offers four data usage methods: user-owned data, user-group shared data, platform-wide shared data, and global sample data. These four data types are stored in different storage spaces, allowing users to select data from the appropriate storage space as needed.

Online Model Development Functionality​

The default container images of the AI platform include JupyterLab functionality. Each user's created development environment comes with JupyterLab, facilitating interactive model development and enabling the quick construction of an independent IDE development environment.

Multiple Connection Methods for Development Environments​

Users can connect to their created development environments via four methods: web-based shell, local shell, VSCode remote connection, and PyCharm remote connection, allowing flexible selection based on usage habits.

Multiple Deep Learning Training Job Modes​

Users can conveniently submit single-node and distributed training jobs via the AI platform, supporting industry-standard deep learning frameworks such as TensorFlow, PyTorch, MXNet, Caffe, and PaddlePaddle.

Automatic Resource Matching​

The AI platform automatically identifies different accelerator types (GPU, MLU, BI, etc.) in the cluster. The scheduler automatically dispatches tasks to the required accelerator type based on business needs.

Automatic Task Fault Tolerance​

The AI platform provides multiple fault tolerance mechanisms for training jobs. It automatically detects errors caused by network interruptions, server downtime, or accelerator loss, and automatically restarts training jobs from checkpoints to ensure reliable execution.