Due to the ubiquitous batch data processing in cloud computing, the fundamental model of scheduling malleable batch tasks and its extensions have received significant attention recently. In this model, a set of n tasks is to be scheduled on C identical machines and each task is specified by a value, a workload, a deadline and a parallelism bound. Within the parallelism bound, the number of the machines allocated to a task can vary over time and its workload will not change accordingly. In this paper, the two core results of this paper are to quantitatively characterize a sufficient and necessary condition such that a set of malleable batch tasks with deadlines can be feasibly scheduled on C machines, and to propose a polynomial time algorithm to produce such a feasible schedule. The core results provide a conceptual tool and an optimal scheduling algorithm to enable proposing new analysis and design of algorithms or improving existing algorithms for extensive scheduling objectives.
In addition, @cite_8 consider DAG-structured malleable tasks and propose two algorithms with approximation ratios of 6 and 2 respectively for the objectives of minimizing the total weighted completion time and the maximum weighted lateness of tasks. The conclusions in @cite_8 also show that scheduling deadline-sensitive malleable tasks is a key to the solutions to scheduling for their objectives. In particular, seeking a schedule for DAG tasks can be transformed into seeking a schedule for tasks with simpler chain-precedence constraints; then whenever there is a feasible schedule to complete a set of tasks by their deadlines, propose an algorithm where each task is completed by at most 2 times its deadline and give two procedures to obtain the near-optimal completion times of tasks in terms of the two scheduling objectives.
@cite_8
236893983
We introduce FlowFlex, a highly generic and effective scheduler for flows of MapReduce jobs connected by precedence constraints. Such a flow can result, for example, from a single user-level Pig, Hive or Jaql query. Each flow is associated with an arbitrary function describing the cost incurred in completing the flow at a particular time. The overall objective is to minimize either the total cost (minisum) or the maximum cost (minimax) of the flows. Our contributions are both theoretical and practical. Theoretically, we advance the state of the art in malleable parallel scheduling with precedence constraints. We employ resource augmentation analysis to provide bicriteria approximation algorithms for both minisum and minimax objective functions. As corollaries, we obtain approximation algorithms for total weighted completion time (and thus average completion time and average stretch), and for maximum weighted completion time (and thus makespan and maximum stretch). Practically, the average case performance of the FlowFlex scheduler is excellent, significantly better than other approaches. Specifically, we demonstrate via extensive experiments the overall performance of FlowFlex relative to optimal and also relative to other, standard MapReduce scheduling schemes. All told, FlowFlex dramatically extends the capabilities of the earlier Flex scheduler for singleton MapReduce jobs while simultaneously providing a solid theoretical foundation for both.