A Review of CUDA, MapReduce, and Pthreads Parallel Computing Models <s> MAPREDUCE <s> We have designed and implemented the Google File System, a scalable distributed file system for large distributed data-intensive applications. It provides fault tolerance while running on inexpensive commodity hardware, and it delivers high aggregate performance to a large number of clients. While sharing many of the same goals as previous distributed file systems, our design has been driven by observations of our application workloads and technological environment, both current and anticipated, that reflect a marked departure from some earlier file system assumptions. This has led us to reexamine traditional choices and explore radically different design points. The file system has successfully met our storage needs. It is widely deployed within Google as the storage platform for the generation and processing of data used by our service as well as research and development efforts that require large data sets. The largest cluster to date provides hundreds of terabytes of storage across thousands of disks on over a thousand machines, and it is concurrently accessed by hundreds of clients. In this paper, we present file system interface extensions designed to support distributed applications, discuss many aspects of our design, and report measurements from both micro-benchmarks and real world use. <s> BIB001 </s> A Review of CUDA, MapReduce, and Pthreads Parallel Computing Models <s> MAPREDUCE <s> MapReduce is a programming model and an associated implementation for processing and generating large data sets. Users specify a map function that processes a key/value pair to generate a set of intermediate key/value pairs, and a reduce function that merges all intermediate values associated with the same intermediate key. Many real world tasks are expressible in this model, as shown in the paper. ::: ::: Programs written in this functional style are automatically parallelized and executed on a large cluster of commodity machines. The run-time system takes care of the details of partitioning the input data, scheduling the program's execution across a set of machines, handling machine failures, and managing the required inter-machine communication. This allows programmers without any experience with parallel and distributed systems to easily utilize the resources of a large distributed system. ::: ::: Our implementation of MapReduce runs on a large cluster of commodity machines and is highly scalable: a typical MapReduce computation processes many terabytes of data on thousands of machines. Programmers find the system easy to use: hundreds of MapReduce programs have been implemented and upwards of one thousand MapReduce jobs are executed on Google's clusters every day. <s> BIB002 </s> A Review of CUDA, MapReduce, and Pthreads Parallel Computing Models <s> MAPREDUCE <s> Bigtable is a distributed storage system for managing structured data that is designed to scale to a very large size: petabytes of data across thousands of commodity servers. Many projects at Google store data in Bigtable, including web indexing, Google Earth, and Google Finance. These applications place very different demands on Bigtable, both in terms of data size (from URLs to web pages to satellite imagery) and latency requirements (from backend bulk processing to real-time data serving). Despite these varied demands, Bigtable has successfully provided a flexible, high-performance solution for all of these Google products. In this paper we describe the simple data model provided by Bigtable, which gives clients dynamic control over data layout and format, and we describe the design and implementation of Bigtable. <s> BIB003
There are many different ways to implement MapReduce and the correct manner depends of the environment that the program will be working in. MapReduce jobs have been created on NUMA multiprocessors, large connection of commodity nodes on a cluster, and shared memory machine models BIB002 . In Fig 12, the MapReduce pseudocode in an implementation of the programming model which strives to succumb to the issue of counting the number of times a word occurs in a document or a set of documents. The map function creates a key/value pair of the name of the word and the initial occurrence of each word which will be 1. The reduce function then sums all the counts that were created by the mapper. There is a MapReduce specification object that takes in arguments of the names of the input and output files associated with the initial data to be computed. Figure. 13 , gives an execution overview of MapReduce and how the programming model has a workflow that initializes a user program which forks tasks to different workers. A Master has the ability to control the flow of forking different tasks/jobs to associated workers. This allows the master to be in control of the designation jobs to different workers and the corrective planning of inevitable fault tolerance issues associated with node failure. The initial data is split or distributed across the nodes on the network/cluster using a unique file system structure BIB001 . The master designates jobs to workers that must find the data associated with their job on the host node, and execute the associate read/write task. The intermediate files are created on the local disks by the map phase. The reduce phase now takes over in which the Master will assign a set of workers to complete the reduce task/jobs. The reduce phase now creates a smaller subset of key/value pairs to be merged together and results sent to the data store BIB003 . The reduce phase then use the API function write to compute a new pair of key/value pairs that now that the sum of the total occurrences of each word in the form of <key, total word occurrence>. Figure 16 depicts the command line arguments needed to compile the java word count file. The input files that are to be parsed in order to count the words in the document are to be stored using the associated file system structured mentioned by Ghemawat, et al. BIB001 . In order for MapReduce to be successfully compiled, an open source version of Google File System, MapReduce, and Bigtable need to be installed. For this example Apache's Hadoop Distributed File System (HDFS), Hadoop Base, and Hadoop MapReduce was installed. The input file was stored using HDFS.