1. Technical Field
This invention relates generally to a multiple-node system having a number of communicatively connected nodes, and more particularly to coherency controllers of each node.
2. Description of the Prior Art
There are many different types of multi-processor computer systems. A Symmetric Multi-Processor (SMP) system includes a number of processors that share a common memory. SMP systems provide scalability. As needs dictate, additional processors can be added. SMP systems usually range from two to 32 or more processors. One processor generally boots the system and loads the SMP operating system, which brings the other processors online. Without partitioning, there is only one instance of the operating system. The operating system uses the processors as a pool of processing resources, all executing simultaneously, where each processor either processes data or is in an idle loop waiting to perform a task. SMP systems increase in speed whenever processes can be overlapped.
A Massively Parallel Processor (MPP) system can use thousands or more processors. MPP systems use a different programming paradigm than the more common SMP systems. In an MPP system, each processor contains its own memory and a copy of the operating system and application, or a portion of the application. Each subsystem communicates with the others through a high-speed interconnect. To use an MPP system effectively, an information-processing problem should be breakable into pieces that can be solved simultaneously. For example, in scientific environments, certain simulations and mathematical problems can be split apart and each part processed at the same time.
A Non-Uniform Memory Access (NUMA) system is a multi-processing system in which memory is separated into distinct banks. NUMA systems are a type of SMP systems. In SMP systems, however, all processors access a common memory at the same speed. By comparison, in a NUMA system, memory on the same processor board or in the same building block or node, as the processor is accessed faster than memory on other processor boards, or in other building blocks or nodes. That is, local memory is accessed faster than distant shared memory. NUMA systems generally scale better to higher numbers of processors than SMP systems.
Each node of a multiple-node system may have one or more controllers that are responsible for managing transactions. For instance, in a NUMA system, the controllers of each node are coherency controllers that are responsible for managing transactions that relate to the local memory of the node, as well as transactions that relate to the local memories of other nodes, which are considered remote memory to the node. Whereas this configuration is operable where each node is partitionable to no more than a single partition, for other configurations, where each physical node encompasses two or more sub-nodes, the coherency controllers of a given physical node may need to process a transaction once for the sending sub-node and once for the receiving sub-node when transactions relate to memory that is local to the physical node but not to the sub-node sending the transaction.
Typically, each sub-node within a physical node has a coherency controller that is responsible for the local memory of the sub-node and memory that is remote to the sub-node. When a transaction relates to memory that is local to the physical node, but not to the immediate sub-node sending the transaction, the transaction is thus processed as a remote transaction by the coherency controller local to the sub-node sending the transaction. The transaction is thus sent to the coherency controller local to the other sub-node that is responsible for the memory that is local to the physical node but not to the sub-node sending the transaction. Such a transaction, then, is processed by two difference coherency controllers local to the physical node. This incurs processing latency resulting from processing by two coherency controllers, which can cause performance degradation.
Alternatively, there may be one coherency controller for all the sub-nodes of a physical node, such that the transactions of all of the sub-nodes are processed by this controller. While a transaction is thus processed by only one coherency controller local to the physical node, and incurs processing latency resulting from processing by just one coherency controller, this latency is nevertheless usually larger than in the situation where there is a coherency controller for each sub-node. This is because the transactions of all the local sub-nodes of the physical node place a greater load on the sole coherency controller, resulting in greater queuing and other types of delays. For these and other reasons, therefore, there is a need for the present invention.