Investigation and Characterization of MapReduce Applications for Big Data Analytics

Abstract

Recently, many organisations have applied Hadoop MapReduce framework for big data analytics. MapReduce applications based on the MapReduce programming model can be developed to process data of large amount. Therefore, understanding a dependency among the resource usage parameters of MapReduce applications is crucially needed from the viewpoint of cloud operators. In this paper, we analyze the inter-dependency of resource usage parameters of MapReduce applications. Autocorrelation of each resource usage parameter and correlation characteristics of each pair of resource usage parameters are investigated. Based on the analysis, we identify several groups of features that can be used to classify MapReduce applications.