The generalized method to have a parallel solution to a computational problem, is to find a way to use Divide & Conquer paradigm in order to have processors acting on its own data and therefore all can be scheduled in parallel. MapReduce is an example of this approach: Input data is transformed by the mappers, in order to feed the reducers that can run in parallel. In general this schema gives efficient problem solutions, but it stops being true when the replication factor grows. We present another program schema that is useful for describing problem solutions, that can exploit dynamic pipeline parallelism without having to deal with replica- tion factors. We present the schema in an example: counting triangles in graphs, in particular when the graph do not fit in memory. We describe the solution in NiMo, a graphical programming language that implements the implicitly functional parallel data ow model of computation. The solution obtained using NiMo, is architecture agnostic and can be deployed in any parallel distributed architecture adapting dynamically the processor usage to input characteristics.
In @cite_3 the Swift T language is described. Is a language implementing the “implicitly parallel functional dataflow” (IPFD) model centered mainly in being able to have as input a variety of data sources. The programming model is very simple and forces the user to think on in-memory data. The language has the ability of integrating leaf tasks written in other languages. Using this feature has proven very effective in implementing solutions in several domains. But the simplicity of the language does not allow some solutions that fully exploit architectures with a massive number of processors. The promote Many-Task Computing. Swift does 4 important things for you: Makes parallelism more transparent implicitly parallel functional dataflow programming Makes computing location more transparent
@cite_3
2077579791
Scientists, engineers, and statisticians must execute domain-specific application programs many times on large collections of file-based data. This activity requires complex orchestration and data management as data is passed to, from, and among application invocations. Distributed and parallel computing resources can accelerate such processing, but their use further increases programming complexity. The Swift parallel scripting language reduces these complexities by making file system structures accessible via language constructs and by allowing ordinary application programs to be composed into powerful parallel scripts that can efficiently utilize parallel and distributed resources. We present Swift's implicitly parallel and deterministic programming model, which applies external applications to file collections using a functional style that abstracts and simplifies distributed parallel execution.