Data parallel processing is a form of computing parallelization across multiple processing units. Data parallel processing is often used to perform workloads that can be broken up and concurrently executed by multiple processing units. For example, to execute a parallel loop routine having 1000 iterations, four different processing units may each be configured to perform 250 different iterations (or different sub ranges) of the parallel loop routine. In the context of data parallel processing, a task can represent an abstraction of sequential computational work that may have a dynamic working size that is typically determined at runtime. For example, a processor can execute a task that processes a sub-range of a parallel loop routine. In some cases, a task may be created by a thread running on a first processor and dispatched to be processed via another thread on a second processor.
Different tasks may be assigned (or offloaded to) various processing units of a multi-core or multi-processor computing device (e.g., a heterogeneous system-on-chip (SOC)). Typically, a task-based runtime system (or task scheduler) determines to which processing unit a task may be assigned. For example, a scheduler can launch a set of concurrently-executing tasks on a plurality of processing units, each unit differently able to perform operations on data for a parallel loop routine.
Multi-processor (or multi-core) systems are often configured to implement data parallelism techniques to provide responsive and high performance software. For example, with data parallel processing capabilities, a multi-core device commonly launches a number of dynamic tasks on different processing units in order to achieve load balancing. Parallel workloads may get unbalanced between various processing units. For example, while multiple processing units may initially get equal sub-ranges of a parallel loop routine, imbalance in execution time may occur. Imbalances in workloads may occur for many reasons, such as that the amount of work per work item is not constant (e.g., some work items may require less work than other work items, etc.); the capabilities of heterogeneous processing units may differ (e.g., big.LITTLE CPU cores, CPU vs. GPU, etc.); rising temperature may throttle frequencies on some processing units more than others, particularly if the heat dissipation is not uniform (as is commonly the case); and other loads may cause some processors to lag more (e.g., loads from other applications, the servicing of system interrupts, and/or the effects of OS scheduling, etc.).
To improve performance while conducting data parallel processing, multi-processor systems may employ work-stealing techniques in which tasks or processing units can be configured to opportunistically take and execute work items originally assigned to other tasks or processing units. For example, when a first processing unit or task finishes an assigned subrange of a shared workload (e.g., a parallel loop routine), the first processing unit may steal additional sub-ranges of work from other processing units/tasks that are still busy processing respective assignments of the shared workload. As load imbalances may often occur, work-stealing operations may allow dynamic load-balancing that improves the utilization of system processing resources and reduces the time to complete parallel work.