BACKGROUND

The present description generally relates to processor topologies using twisted torus configurations

BACKGROUND

Machine learning often requires large amounts of computation and communication bandwidth. To provide the needed processing capability, devices such as artificial intelligence (AI) accelerators can be used. In some cases, networks of many interconnected AI accelerators can be used to provide the desired processing capability. More generally, clusters of processors can have different sizes and configurations, resulting in different properties and thus varying levels of suitability for different computational tasks.

SUMMARY

In some implementations, a system provides a large cluster of processing nodes that are connected by a reconfigurable interconnect fabric. The reconfigurable interconnect network can include switching devices, e.g., multiplexers or other elements, to selectively enable various combinations of direct, physical-layer connections between processing nodes. The processing nodes can be artificial intelligence (AI) accelerator devices or machine learning (ML) accelerator devices, for example, application-specific integrated circuits (ASICs) such as a Tensor Processing Unit (TPU). As a result, the system can provide a large cluster of accelerators that are configurable in many different ways to achieve the performance characteristics needed for different tasks.

The system can be implemented as a shared system that allows different users to concurrently and remotely access different portions of a large cluster of computing nodes. For example, a host system can dynamically allocate groups of processing nodes from a cluster of processing nodes that may include, for example, hundreds or thousands of different processors. The groups of nodes can be different, distinct proper subsets of the processing nodes in the system. Different users and tasks have different requirements, and so different numbers of processing nodes can be allocated for different situations. For example, different users may respectively request different numbers of nodes, typically as powers of two, e.g., 2, 4, 8, 16, 32, 64, 128, etc. In addition to allowing different numbers of nodes to be allocated, the system can organize the nodes in different topologies using the reconfigurable interconnect fabric, e.g., with different arrangements of data connections between the nodes. The different networks (e.g., groups of nodes, potentially with different node interconnection scheme), are then made available independently to different users. In effect, a single large physical cluster of processing nodes can be presented to multiple different users as virtual clusters configured according to the needs of the different users. Each group of nodes forming a virtual cluster operates separately from the groups of nodes for other users, allowing each to be isolated from the rest of the network.

One of the benefits of the system is the ability to use different interconnection schemes for different node topologies, to improve the performance of computing tasks performed with different sizes and topologies of networks. For networks that have a cubical arrangement of nodes, a torus configuration provides a symmetric network with high performance. For other network configurations that are not cubical, however, the a twisted torus interconnection scheme can be used. This can provide symmetry to network topologies that would not have been symmetrical with a standard torus. For non-cubical dimensions of nodes, the twisted torus interconnect can offer higher bisection bandwidth, better load balance characteristics, and lower network diameter. For example, interconnecting a group of nodes in a 8×4×4 configuration as a 3D twisted torus offers a ˜1.73× increase in effective bandwidth for all-to-all (e.g., uniform random) traffic compared to a traditional 3D torus interconnect.

In some implementations, when a user launches a machine learning training job through a request to the system, the host loads the training data from the storage and set up the environment. This can include selecting an appropriate number of nodes, selecting a topology for the nodes, and selecting an interconnect scheme for the nodes (e.g., whether to add a twist to the torus, and if so, to what extent). The system then allocates the network with the selected parameters and makes it available for performing the user's processing task. After the network is established, the ASIC accelerator chips in the network perform the needed tasks, such as training machine learning models (e.g., neural networks) and communicate with each other through fast, direct connections such as Inter-Core Interconnect (ICI) links. Tasks such as machine learning model training may require a duration on the order from seconds up to days, depending on the machine learning model size, training data size, and the number of ASIC accelerator chips. During that period, the amount of inter-host communication is much less than the ASIC-ASIC communications between processing nodes of an allocated network.

In some implementations, a method performed by one or more computers includes: providing a cluster of processing nodes coupled using a reconfigurable interconnect fabric; determining a number of processing nodes to allocate as a network within the cluster and a topology for the network; selecting an interconnection scheme for the network, wherein the interconnection scheme is selected from a group that includes at least a torus interconnection scheme and a twisted torus interconnection scheme; allocating the determined number of processing nodes of the cluster in the determined topology; setting the reconfigurable interconnect fabric to provide the selected interconnection scheme for the processing nodes in the network; and providing access to the network for performing a computing task.

In some implementations, selecting the interconnection scheme comprises selecting between the torus interconnection scheme and the twisted torus interconnection scheme based on the determined number of processing nodes.

In some implementations, selecting the interconnection scheme comprises selecting the torus interconnection scheme based on determining that, for a network of a number of dimensions that the reconfigurable interconnect fabric supports, the number of processing nodes allows the network to have equal size in each of the dimensions. For example, if the reconfigurable interconnect fabric enables torus networks of two dimensions or three dimensions, the system can determine if the number is a perfect square or a perfect cube.

In some implementations, the selected topology has a first size in a first dimension and a second size in a second dimension. Selecting the interconnection scheme comprises selecting the twisted torus interconnection scheme based on determining that the first size is a multiple of the second size.

In some implementations, the selected topology for the network comprises an arrangement of nodes that extends along multiple dimensions and includes multiple nodes along each of the multiple dimensions, wherein the selected topology has different amounts of nodes along at least two of the multiple dimensions. Selecting the interconnection scheme comprises selecting the twisted torus interconnection scheme such that the network is symmetric.

In some implementations, the twisted torus interconnection scheme includes wraparound connections made using switching elements of the reconfigurable interconnect fabric. Wraparound connections connect nodes or edges of the network that face opposite directions along a same dimension, wherein the wraparound connections for a first dimension in which the network is longest do not include any offsets in other dimensions, and wherein the wraparound connections for a second dimension in which the network is shorter than the first dimension has an offset in the first dimension.

In some implementations, the wraparound connections for the second dimension are each determined by connecting a starting node with an ending node that has: (i) a position in the second dimension that is the same as the starting node, and (ii) a position in the first dimension that is equal to a result of a modulo operation involving (a) a sum of a position of the starting node in the first dimension and a predetermined twist increment determined based on the node topology and (b) a length of the longest dimension.

In some implementations, selecting the interconnection scheme comprises selecting the twisted torus interconnection scheme. The method includes selecting an amount of twist and dimensions in which to apply an offset for the twist based on lengths of the selected topology.

In some implementations, the selected topology is a two-dimensional topology.

In some implementations, the selected topology is a three-dimensional topology.

In some implementations, the cluster of processing nodes comprises multiple segments having a predetermined size and arrangement of multiple processing nodes, the segments having mesh connections between the nodes in each segment. The reconfigurable interconnect fabric comprises switching elements to permit dynamic, programmable reconfiguration of connections for external-facing data ports of processing nodes in each segment.

In some implementations, the segments are each a 4×4×4 group of processing nodes.

In some implementations, the processing nodes are each separate application specific integrated circuits (ASICs).

In some implementations, the reconfigurable interconnect fabric comprises switches for data-carrying optical signals.

In some implementations, the computing task comprises training a machine learning model.

In some implementations, the method includes storing multiple configuration profiles specifying different configurations of the reconfigurable interconnect fabric to connect subsets of the processing nodes in the cluster; and selecting a configuration profile from among the multiple configuration profiles. Switching elements of the reconfigurable interconnect fabric are set according to the selected configuration profile.

In some implementations, the method includes initializing routing tables for processing nodes in the network based on stored routing information corresponding to the selected configuration profile.

In some implementations, the method includes allocating distinct networks of processing nodes for different users from among the processing nodes in the cluster, the distinct networks having different interconnection schemes.

Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods. A system of one or more computers can be configured to perform particular operations or actions by virtue of having software, firmware, hardware, or a combination of them installed on the system that in operation causes or cause the system to perform the actions. One or more computer programs can be configured to perform particular operations or actions by virtue of including instructions that, when executed by data processing apparatus, cause the apparatus to perform the actions.

The details of one or more embodiments of the invention are set forth in the accompanying drawings and the description below. Other features and advantages of the invention will become apparent from the description, the drawings, and the claims.

