FIELD OF THE INVENTION

The present invention generally relates to data processing, in particular to a hybrid network system, a communication method and a network node, which can be applied to, for example, parallel computation, data centers, distributed file systems and the like.

BACKGROUND OF THE INVENTION

Many computer fields highly depending on communication have the demands of improving the communication performance, for example, parallel computation, data centers and distributed file systems, etc.

Parallel computation is an evolving computer field, and the parallel computation refers to executing the same task on a plurality of processors (the task is decomposed and specially adjusted) to obtain a result more quickly. The parallel computation is based on the following facts: a process of solving a problem can be generally divided into a plurality of smaller tasks, and these smaller tasks can be executed simultaneously by coordination.

A parallel computer executes a parallel algorithm. The parallel algorithm is decomposed into many small parts of tasks, these small parts of tasks are executed on many different processing devices and are finally summarized together to obtain a data processing result. In the specification, the numerous processing devices for executing the small parts of a parallel program are called “computing nodes”, and the parallel computer is composed of the computing nodes and other processing nodes (e.g., input/output nodes and service nodes).

In order to execute the parallel program, the nodes of the parallel computer often need to carry out a lot of data communication. Typically, the common mode of the nodes for carrying out the data communication is message passing.

As one of the main components of the parallel program, the communication influences the performance of the parallel program to a larger extent.

Currently, MPI (“message passing interface (Message Passing Interface)”) is a fact standard for communication of the computing nodes executing the parallel program on the parallel computer. The MPI is an existing parallel communication library and is a module of computer program instructions which carrie out data communication on the parallel computer. The MPI is released by an MPI forum, and the MPI forum is an open group and has many organization representatives that define and maintain the MPI standard.

A communication function of the parallel program mainly includes a point-to-point communication function and a collective communication function. The point-to-point communication function can completely execute the data exchange between two parallel processes, including blocking communication (MPI_Send, MPI_Recv) and non-blocking communication (MPI_Isend, MPI_Irecv) and the like; and the collective communication function can realize the data exchange of a plurality of processes (process groups), including MPI_Barrier, MPI_Bcast, MPI_Allgather, MPI_Alltoall or the like. In this specification, the term “point-to-point communication function” and the term “point-to-point operation” are interchangeable, and the “collective communication function” are interchangeable with the term “collective operation” (also referred to as “collective action”, “group communication” and the like in the field sometimes).

Inventors have found that the expansion of the parallel computing capability of the parallel computer is often limited by the data communication performance. With the expansion of the scale, the proportion of communication time in total execution time increases, which is mainly caused by the communication time of the collective communication function, for example, the computation time only occupies 39.2% in 2048 processes, and to improve the expandability of the application, the communication performance of the system, particularly the collective communication performance, must be improved.

The inventors have also found that in an operating system mirror broadcast application of a cloud platform of the data center and a file backup application of the distributed file system, the communication performance also needs to be improved.

SUMMARY OF THE INVENTION

In view of the above conditions, the present invention is proposed.

Various embodiments of the present invention provide the following technical solutions:

according to one aspect of the present invention, a hybrid network system is provided, including: a combined network system architecture of a point-to-point network and a broadcast network, wherein the point-to-point network is provided with a plurality of point-to-point channels, each network node can send a message to a corresponding network node or receive the message from the corresponding network node through the point-to-point channel, and wherein the broadcast network is provided with a plurality of broadcast channels, and each broadcast channel can only have one sender at most at the same time, but can have a plurality of receivers; at least a part of network nodes have the capability of sending messages to be broadcast through the broadcast channels, at least a part of network nodes have the capability of receiving the broadcast messages, wherein at least some network nodes are configured to select between the broadcast channels and the point-to-point channels at least according to communication tasks to be executed per se and complete the communication tasks to be executed through the selected channels.

According to another aspect of the present invention, a communication method in a hybrid network system is provided, the hybrid network system can include a combined network system architecture of a point-to-point network and a broadcast network, wherein the point-to-point network is provided with a plurality of point-to-point channels, each network node can send a message to a corresponding network node or receive the message from the corresponding network node through the point-to-point channel, and wherein the broadcast network is provided with a plurality of broadcast channels, and each broadcast channel can only have one sender at most at the same time, but can have a plurality of receivers; at least a part of network nodes have the capability of sending messages to be broadcast through the broadcast channels, at least a part of network nodes have the capability of receiving the broadcast messages, and the communication method can include: at least some network nodes select between the broadcast channels and the point-to-point channels at least according to communication tasks to be executed and complete the communication tasks to be executed through the selected channels.

According to yet another aspect of the present invention, a network node in a hybrid network system is provided, the hybrid network system can include a combined network system architecture of a point-to-point network and a broadcast network, the point-to-point network is provided with a plurality of point-to-point channels, the broadcast network is provided with a plurality of broadcast channels, and each broadcast channel can only have one sender at most at the same time, but can have a plurality of receivers; the network node can send a message to a corresponding network node or receive the message from the corresponding network node through the point-to-point channel, and has the capability of sending a message to be broadcast and receiving the broadcast message through the broadcast channel, and the network node is configured to select between the broadcast channels and the point-to-point channels at least according to a communication task to be executed per se and complete the communication task to be executed through the selected channel.

By adopting the hybrid network system according to the embodiments of the present invention, the network architecture simultaneously provided with the point-to-point network and the broadcast network can select the point-to-point network or the broadcast network to complete communication according to the communication needing to be executed for the specific task to be executed, therefore compared with the traditional simple point-to-point network architecture, flexibility can be brought, the communication times are decreased, the necessary communication time is shortened, the pressure of point-to-point communication network resources is relieved, and the application expandability is improved.

A parallel computer adopting the hybrid network architecture according to the embodiments of the present invention and a collective operation execution method adopt the system structure of the point-to-point network and the broadcast network, and at least a part of broadcast or multicast communication in a collective operation is completed by at least a part of broadcast networks, therefore the collective communication performance can be significantly improved, and the expandability of parallel computation is improved.

With the hybrid network applied to a data center according to the embodiments of the present invention, a mirror can be sent through the broadcast network, so as to greatly reduce the network overhead in virtual machine deployment.

With the hybrid network applied to a distributed file system according to the embodiments of the present invention, at least three transcripts can be completed by transmitting file contents through the broadcast network at one time, so that the communication performance is greatly improved, and the computing resources are saved.

