1. A method of handling exceptions in a data parallel system, comprising: forwarding exceptions thrown by concurrent worker tasks to a coordination task; aggregating the thrown exceptions into an aggregation exception structure; determining whether the aggregation exception structure will be handled by an exception handler; and unwinding the concurrent worker tasks when it is determined that the aggregation exception structure will be handled; preventing unwinding of the concurrent worker tasks when it is determined that the aggregation exception structure will not be handled; providing a notification to the concurrent worker tasks when it is determined that the aggregation exception structure will not be handled; searching for an exception handler for the exceptions thrown by the concurrent worker tasks when the concurrent worker tasks are provided the notification.