In developments of emerging technology, such as new wireless standards or Artificial Intelligence, the amount of data required to be processed is increasing substantially. With the profusion of data, more computational requirements are placed on general purpose CPUs, specialized CPUs (i.e. GPU, TPU) and/or specialized Hardware Accelerators to expeditiously process the data.
As the computational requirement placed on the processors increases, the performance of the processors is often inadequate to handle computationally intensive tasks on large amount of data. In some cases, even if specialized processors are capable of handling the computational requirements, the cost of such processors is often prohibitive for many applications.
There are various factors which limit the computational capabilities of a processor. Traditionally, the processors use internal registers to temporarily hold the source input data which are loaded from the data memory. The processor then performs an arithmetic or other programmed operation using the values stored in the temporary registers as the operands, and writes the result of the operation to another temporary register. Finally, the processor stores the result in the temporary register back to the data memory.
For performing such operations, many instructions are required. For example, ADD Immediate instructions to calculate the operand addresses; LOAD Instructions to load the operands; MULTIPLY instruction to multiply the operands; ADD Immediate instruction to calculate the destination address; and STORE instruction to write the result to the destination memory location.
During the execution of these instructions, due to the inherent load/store latency associated with the data memory and the limited availability of the temporary registers, the instruction executions are often blocked by pipeline stalls resulting in degraded processor performance. The problem of pipeline stall is compounded when the processor operates on large sets of data.
Other common techniques employed in the industry, such as SIMD and Vector Instruction Extensions, try to address the performance issue by parallel data processing. However, these techniques, even though they obtain performance increase through parallelism, are still subject to the aforementioned limitations.
Therefore, it is desirable to have a flexible solution capable of processing large amount of data, which can also be quickly programmed, deployed, and modified as the product matures.