TECHNICAL FIELD

The present disclosure relates to a gateway for use in a computer system to interface a host with a subsystem for acting as a work accelerator, and in particular to the provision of data to the accelerator for processing.

BACKGROUND

In the context of processing data for complex or high volume applications, a work accelerator may be a subsystem to which processing of certain data is offloaded from a host system. Such a work accelerator may have a specialised hardware for performing specific types of processing.

As an example, one area of computing in which such a specialised accelerator subsystem may be of use is found in machine intelligence. As will be familiar to those skilled in the art of machine intelligence, a machine intelligence algorithm is based around performing iterative updates to a “knowledge model”, which can be represented by a graph of multiple interconnected nodes. The implementation of each node involves the processing of data, and the interconnections of the graph correspond to data to be exchanged between the nodes. Typically, at least some of the processing of each node can be carried out independently of some or all others of the nodes in the graph, and therefore large graphs expose great opportunities for multi-threading. Therefore, a work accelerator specialised for machine intelligence applications may comprise a large degree of multi-threading. One form of parallelism can be achieved by means of a processor comprising an arrangement of multiple tiles on the same chip (i.e. same die), each tile comprising its own separate respective processing unit and memory (including program memory and data memory). Thus separate portions of program code can be run in parallel on different ones of the tiles. The tiles are connected together via an on-chip interconnect which enables data to be exchanged between them. Such an accelerator may function as a subsystem for a host system to perform parallel processing of data sets provided to it.

In general, there may exist dependencies between the portions of a program running on different tiles. A technique is, therefore required to prevent a piece of code on one tile running ahead of data upon which it is dependent being made available by another piece of code on another tile. There are a number of possible schemes for achieving this, one of which is described here by way of example, ‘BSP’, bulk synchronous parallel. According to BSP, each tile performs a compute phase and an exchange phase in an alternating cycle. During the compute phase each tile performs one or more computation tasks locally on tile, but does not communicate any results of its computations with any others of the tiles. In the exchange phase each tile is allowed to exchange one or more results of the computations from the preceding compute phase to and/or from one or more others of the tiles in the group, but does not yet proceed to the next compute phase. Further, according to the BSP principle, a barrier synchronization is placed at the juncture transitioning from the compute phase into the exchange phase, or transitioning from the exchange phase into the compute phase, or both. That is to say, either: (a) all tiles are required to complete their respective compute phases before any in the group is allowed to proceed to the next exchange phase, or (b) all tiles in the group are required to complete their respective exchange phases before any tile in the group is allowed to proceed to the next compute phase, or (c) both. In some scenarios a tile performing computation may be allowed to communicate with other system resources such as a network card or storage disk, as long as no communication with other tiles in the group is involved.

During an exchange phase, data exchange may not only take place internally (i.e. between tiles) within an accelerator, but in some circumstances may be required to take place between an accelerator and external storage, e.g. a host system. When a subsystem acts as a work accelerator, it is configured to process data sets provided to it (e.g. from a host system). The data sets should advantageously be retrieved such that they can be provided to the accelerator at the appropriate precompiled data exchange synchronization point, which may represent a barrier synchronization.

When scaling subsystems by connecting them together—directly or indirectly—a problem may occur, which is that the data for a subsystem may not be made available to the subsystem in a timely fashion. This may result in a longer exchange phase being required, as a greater amount of time is required to deliver the appropriate data to the subsystem.

SUMMARY OF THE INVENTION

The present invention uses the concept of a gateway which can be used to provide data to the accelerators from external storage and thus interconnect them to scale a subsystem acting as a work accelerator. The accelerator receives the data from the gateway in an exchange phase and computes the data in a following compute phase. In some forms of the gateway, the gateway itself is an active processor of data and autonomously manages its data flows. The gateway acts as an intermediary between external storage and one or more accelerators. The gateway has a local memory for temporarily storing data for delivery to the accelerator from the external storage. This raises the challenge of how to ensure that the gateway makes available the data to the accelerator in a timely fashion during an exchange phase.

According to a first aspect, there is provided a computer system comprising: (i) a computer subsystem configured to act as a work accelerator, and (ii) a gateway connected to the computer subsystem, the gateway enabling the transfer of data to the computer subsystem from external storage at pre-compiled data exchange synchronization points attained by the computer subsystem, which act as a barrier between a compute phase and an exchange phase of the computer subsystem, wherein the computer subsystem comprises a plurality of processing units and a plurality of memories associated with the processing units, at least one of the memories including a first compiled code sequence comprising at least one instruction executable by at least one of the plurality of processing units to pull data from a gateway transfer memory of the gateway in response to the pre-compiled data exchange synchronization point attained by the subsystem, wherein the gateway comprises at least one processor configured to perform at least one operation to pre-load at least some of the data from a first memory of the gateway to the gateway transfer memory in advance of the pre-compiled data exchange synchronization point attained by the subsystem.

In some embodiments, the data to be pulled from the gateway transfer memory belongs to a plurality of streams.

In some embodiments, the gateway transfer memory comprises a plurality of buffers, wherein each of the buffers is configured to store data belonging to an associated one of the plurality of streams.

In some embodiments, each of the buffers is a virtual data buffer, wherein at least one of the virtual data buffers store data in a physically discontiguous space in the gateway transfer memory.

In some embodiments, the first compiled code sequence is configured to cause only one of the plurality of processors to issue read requests to pull data of a first of the plurality of streams from the gateway transfer memory.

In some embodiments, the at least one processor is configured to, in advance of the pre-compiled data exchange synchronization point attained by the subsystem, pre-load data to be pulled from the gateway transfer memory in response to each of a plurality of upcoming pre-compiled data exchange synchronization points attained by the subsystem.

In some embodiments, the computer subsystem is configured to execute the at least one instruction to pull data from the gateway transfer memory by issuing at least one read request to the gateway.

In some embodiments, the gateway is configured to pre-load some of the data, wherein the gateway is configured to receive the at least one read request and, in response to the at least one read request, load remaining data into the gateway transfer memory from the first memory to be pulled from the gateway transfer memory in response to the pre-compiled data exchange synchronization point attained by the subsystem.

In some embodiments, the at least one read request comprises at least one of: an address of the first memory; and a number of bytes to be pulled from the gateway transfer memory.

In some embodiments, the first compiled code sequence comprises at least one instruction executable by the computer subsystem to pull a second set of data from the first memory in response to the pre-compiled data exchange synchronization point attained by the subsystem.

In some embodiments, the least one processor is configured to pre-load data of a first data stream from the first memory to the gateway transfer memory in advance of the pre-compiled data exchange synchronization point attained by the subsystem, wherein the second set of data pulled from the first memory comprises data of a second data stream.

In some embodiments, the at least one processor of the gateway is configured to: check whether memory availability requirements are met for pre-loading the at least some of the data and the second set of data into the gateway transfer memory; and pre-load the at least some of the data from a first memory of the gateway to the gateway transfer memory in response to determining that the memory availability requirements are met for pre-loading the at least some of the data.

In some embodiments, the at least one processor of the gateway comprises a field programmable gate array.

In some embodiments, the gateway comprises at least one instruction memory configured to store a second compiled code sequence expressing the at least one operation, wherein the first and second compiled code sequences are generated as a related set at compile time.

In some embodiments, the gateway comprises a streaming engine configured to execute a set of data transfer instructions to stream data through the gateway from the external storage to the computer subsystem, wherein the streaming engine comprises the at least one processor.

In some embodiments, the computer subsystem is configured to, in response to attaining the pre-compiled data exchange synchronization point, transmit a synchronization request to the gateway, wherein the gateway is configured to, in response to receiving the synchronization request, transmit a synchronization acknowledgment to the computer subsystem, wherein the computer subsystem is configured to pull the at least some of the data from the gateway transfer memory in response to receiving the synchronization acknowledgement.

In some embodiments, the gateway is configured to: store a number of credits indicating the availability of data for transfer to the computer subsystem at each pre-compiled data exchange synchronization point; and transmit the synchronization acknowledgment to the computer subsystem in response to determining that the number of credits comprises a non-zero number of credits.

In some embodiments, the gateway is configured to interface the computer subsystem with a host to enable the computer subsystem to act as a work accelerator to the host, wherein the system comprises an accelerator interface configured to connect the computer subsystem to the gateway to enable the transfer of the data from the gateway to the computer subsystem.

