BACKGROUND

A data center commonly contains a fleet of servers with various processing devices, such as graphics processing units (GPUs) and accelerators connected to the servers. The processing devices may be allocated to various tasks, such as high performance computing, cloud gaming, virtual machines, video encoding, Infrastructure-as-a-service deployment, and so on. The performance of a given task may involve processing at multiple processing devices, each being connected to the server. In addition to the server being capable of functioning as a host and communicating to the processing devices, the processing devices may also be capable of peer-to-peer communication between one another, that is, without communication being relayed through the server.

Data center server fleets can contain hundreds or even thousands of servers with varying numbers of GPUs and accelerators connected to them. As such, the bandwidth and power requirements of a server fleet can be significant. Accordingly, it is necessary to carefully manage the bandwidth and power requirements of the server fleet.

BRIEF SUMMARY

The present disclosure provides methods and system for power management of a data center server fleet, while ensuring sufficient bandwidth is provided for all tasks assigned to the server fleet. In order to provide for these benefits, information about the processing devices connected to each server of the server fleet may be obtained, and this information may be stored and accessed in order to determine to which one or more processing devices a task is assigned. Candidate groups of processing devices may be a non-exhaustive list. Altogether, the obtained information is used to narrow down a list of candidates for all received tasks, to further narrow down a list of candidates for any one given task, and to further select a single group from among the candidates to perform the task. The obtained information may be regularly updated to reflect changes in the server fleet properties. The obtained information may also be updated regularly as tasks are assigned, completed, or both.

One aspect of the present disclosure is directed to a method including receiving, by one or more processors, a first request to allocate a first task to one or more of a plurality of processing devices, each processing device connected to a corresponding host server, at least some processing devices configured to communicate with one another via a peer-to-peer connection, the first request indicating a required bandwidth for performing the first task, accessing, by the one or more processors, one or more lists of predefined processing device groups, each processing device group including one or more processing devices connected to a common corresponding host server, each list indicating availability of processing device groups of a respective server for allocation of tasks, and further indicating available bandwidth for each available processing device group, assigning, by the one or more processors, the first task to a first processing device group having an available bandwidth greater than or equal to the required bandwidth for performing the first task, and updating, by the one or more processors, the list including the first processing device group to indicate that each of the first processing device group and any other processing device group sharing at least one processing device with the first processing device group is unavailable.

In some examples, updating the list may include indicating that the first processing device group is allocated and that the any other processing device group sharing at least one processing device with the first processing device group is unavailable.

In some examples, the method may further include receiving, by the one or more processors, a second request to allocate a second task to one or more of the plurality of processing devices, the second request indicating a required bandwidth for performing the second task, accessing, by the one or more processors, one or more lists including the updated list, assigning, by the one or more processors, the second task to a second processing device group having an available bandwidth greater than or equal to the required bandwidth for performing the second task, wherein the first processing device group and the second processing device group do not share any processing devices, and re-updating, by the one or more processors, the updated list to indicate that each of the second processing device group and any other processing device group sharing at least one processing device with the second processing device group is unavailable.

In some examples, the method may further include, receiving, by the one or more processors, an indication that the first task is completed, and updating, by the one or more processors, the list including the first processing device group to indicate that the first processing device group is available, and that any other processing device group sharing at least one processing device with the first processing device group and not sharing a processing device with another allocated processing device group is available.

In some examples, updating the list may include updating the available bandwidth for one or more processing device groups sharing at least one bus connection with the first processing device group. For a given processing device group for which the available bandwidth is updated, the updated available bandwidth may be either a difference between the available bandwidth prior to the update and the required bandwidth for performing the first task, or a measured amount of available bandwidth while the first task is being performed.

In some examples, the list may include the first processing device group further indicates, for each available processing device group, an amount of additional power needed for allocating the first task to the available processing device group, and the method may further include designating an available processing group having a lowest amount of power needed as the first processing device group. Updating the list including the first processing device group may include updating the amount of additional power needed for one or more processing device groups sharing at least one hardware component or mechanical component with the first processing device group. For a given processing device group for which the amount of additional power needed is updated, the updated amount of additional power needed may be determined based on predetermined power requirements for the shared at least one hardware component or mechanical component.

In some examples, the first request may indicate each of a required host server bandwidth and a required peer-to-peer bandwidth for performing the first task. The list including the first processing device group may indicate each of an available host server bandwidth and an available peer-to-peer bandwidth for each available processing device group. Assigning the first task to the first processing device group may be further based on an available host server bandwidth of the first processing group being greater than or equal to the required host server bandwidth for performing the first task, and an available peer-to-peer bandwidth of the first processing group being greater than or equal to the required peer-to-peer bandwidth for performing the first task.

Another aspect of the disclosure is directed to a method including receiving, by one or more processors, a request to allocate a task to one or more of a plurality of processing devices, each processing device connected to a corresponding host server, at least some processing devices configured to communicate with one another via a peer-to-peer connection, the request indicating a required bandwidth for performing the task, receiving, by the one or more processors, information for a plurality of processing device groups, each processing device group including one or more processing devices connected to a common corresponding host server, the information indicating, for each given processing device group, an available bandwidth of the given processing device group and an amount of additional power needed for activating the given processing device group, selecting, by the one of more processors, from a subset of processing device groups having an available bandwidth greater than or equal to the required bandwidth for performing the task, a processing device group having a lowest amount of additional power needed for assigning the task, and transmitting, by the one or more processors, a request to assign the task to the selected processing device group.

In some examples, the method may further include updating the information for the plurality of processing device groups based on assignment of the task to the selected processing device group. Updating the information may include updating, for each unselected group of the plurality of processing device groups, the amount of additional power needed for activating the unselected processing device group.

In some examples, the request may further indicate a group size parameter, the information may indicate, for each given processing device group, a group size of the given processing device group, and the processing device group having the lowest amount of additional power needed for assigning the task may be selected from a subset of processing device groups having an available bandwidth greater than or equal to the required bandwidth for performing the task and having a group size within the group size parameter. The group size of the given processing device group may indicate a total number of accelerators, graphical processing units, or both, included in the given processing device group.

In some examples. requesting to assign the task to the selected processing device group may include transmitting, by the one or more processors, to the host server of the selected processing device group instructions for initiating the task.

Yet another aspect of the disclosure is directed to a method including storing, by one or more processors of a server, information about a plurality of processing devices connected to and controlled by the server, wherein at least some of the plurality of processing devices are configured to communicate with one another via a peer-to-peer connection, wherein the stored information includes identifications of subgroups of the plurality of processing devices, each identified subgroup associated in the stored information with a respective indication of availability, a respective indication of available bandwidth, and a respective indication of power required for activation, receiving, by the one or more processors of the server, a request to assign a task to a specified subgroup, assigning, by the one or more processors of the server, the task to the specified subgroup, and updating, by the one or more processors of the server, the stored information to indicate changes in availability, available bandwidth, and power required for activation for at least some of the plurality of processing devices in response to the task being assigned to the specified subgroup.

The method may further include receiving, by the one or more processors of the server, a request for the stored information from a requesting node, and transmitting, by the one or more processors of the server, the stored information to the requesting node in response to the request. The request to assign the task may be received from the requesting node.

In some examples, the method may further include collecting the information about the plurality of processing devices, wherein information is collected for a non-exhaustive list of subgroups of the plurality of processing devices, wherein subgroups are excluded from the non-exhaustive list based at least in part on a quantity of processing devices included in the excluded subgroups. Storing the information about the plurality of processing devices may involve storing the compiled information. The non-exhaustive list of subgroups includes a maximum number of subgroups for each quantity of processing devices up to a predetermined maximum quantity, and the method may further include, for each subgroup of processing devices within the predetermined maximum quantity, determining a number of links connecting the processing devices of the subgroup, and for each quantity of processing devices up to the predetermined maximum quantity, selecting up to the maximum number of subgroups having a fewest number of links.

Yet a further aspect of the disclosure is directed to a system including a data center including a plurality of host servers. Each host server may be connected to a separate group of processing devices. At least some processing devices may be configured to communicate with one another via peer-to-peer connections. Each host server including grouping data identifying subgroups of the plurality of processing devices, each identified subgroup being associated with a respective indication of availability, a respective indication of available bandwidth, and a respective indication of power required for activation in the grouping data. The system may further include memory for storing instructions. The system may further include one or more processors coupled to the memory and configured to execute the stored instructions.

Executing the instructions may involve performing any of the methods described in any one or combination of the embodiments herein.

