TECHNICAL FIELD

The present disclosure generally relates to scalable query processing using parallel processing.

BACKGROUND

As the world becomes more data driven, database systems and other data systems are storing more and more data. For a business to use this data, different operations or queries are typically run on this large amount of data. Some operations, such as those including large table scans, can take a substantial amount of time to execute on a large amount of data. The time to execute such operations can be proportional to the number of computing resources used for execution, so time can be shortened by using more computing resources.

To this end, some data systems can provide a pool of computing resources, and those resources can be assigned to execute different operations. However, in such systems, the assigned computing resources typically work in conjunction (for example as a process group). Hence, their assignments are fixed and static. Because the assigned computing resources work together to execute the operation, they are typically assigned their respective part at the outset, e.g., roles and assignments are fixed ahead of time. Therefore, these systems are not flexible regarding dynamic changes in the number of available resources.

Moreover, these systems cannot track the performance of all of the computing resources. Therefore, in the event of an error with one of the computing resources, the entire job would have to be re-performed, wasting time and resources.

