Storage Resources
Storage Architecture
The AIStation platform uses Huawei DPC (Distributed Parallel Client) as the backend storage system. Running as a storage client on compute nodes, it can connect to multiple storage nodes simultaneously, providing standard POSIX and MPI-IO interfaces to upper-layer applications for better compatibility and superior performance.
Advantage 1: DPC parallel client supports MPI-IO, better adapting to the HPC ecosystem.
Advantage 2: A single DPC parallel client can connect to multiple storage nodes, achieving higher single-client, single-stream performance.
Advantage 3: The DPC parallel client has a shorter I/O path, with the client directly distributing data.
Advantage 4: The DPC parallel client offers superior load balancing capabilities, with I/O-level load balancing.
Storage Performance
Read/Write Performance
| Number of Nodes | Wite(MiB/s) | Read(MiB/s) |
|---|---|---|
| 1 | 13041 | 0490 |
| 5 | 55062 | 3914 |
| 10 | 114598 | 86357 |
| 20 | 215547 | 40452 |
| 50 | 272937 | 49231 |
| 100 | 269529 | 39375 |
IOPS Performance
| Number of Nodes | Wite | Read |
|---|---|---|
| 1 | 444651 | 188394 |
| 5 | 1479884 | 862600 |
| 10 | 2543510 | 1917108 |
| 20 | 4849974 | 3638827 |
| 50 | 7912825 | 5245717 |
| 100 | 10071958 | 8582061 |
mdtest Performance
| Number of Nodes | File creation | File stat | File read | File removal |
|---|---|---|---|---|
| 1 | 111373 | 551635 | 630209 | 86979 |
| 5 | 442197 | 2353586 | 2243375 | 315899 |
| 10 | 726037 | 4707264 | 4031472 | 540862 |
| 20 | 783964 | 9388048 | 9238753 | 731609 |
| 50 | 713123 | 21922473 | 11214082 | 674313 |
| 100 | 700855 | 44213875 | 24049485 | 333082 |