A Review on Business Intelligence and Big Data <s> Discussion <s> BI technologies are essential to running today's businesses and this technology is going through sea changes. <s> BIB001 </s> A Review on Business Intelligence and Big Data <s> Discussion <s> This paper demonstrates ETLMR, a novel dimensional Extract--Transform--Load (ETL) programming framework that uses Map-Reduce to achieve scalability. ETLMR has built-in native support of data warehouse (DW) specific constructs such as star schemas, snowflake schemas, and slowly changing dimensions (SCDs). This makes it possible to build MapReduce-based dimensional ETL flows very easily. The ETL process can be configured with only few lines of code. We will demonstrate the concrete steps in using ETLMR to load data into a (partly snowflaked) DW schema. This includes configuration of data sources and targets, dimension processing schemes, fact processing, and deployment. In addition, we also present the scalability on large data sets. <s> BIB002 </s> A Review on Business Intelligence and Big Data <s> Discussion <s> This paper explores the convergence of Data Warehousing, OLAP and data-intensive Cloud Infrastructures in the context of so-called analytics over Big Data. The paper briefly reviews some state-of-the-art proposals, highlights open research issues and, finally, it draws possible research directions in this scientific field. <s> BIB003 </s> A Review on Business Intelligence and Big Data <s> Discussion <s> Evidence based medicine (EBM) is the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients. Each year, a significant number of research studies (potentially serving as evidence) are reported in the literature at an ever-increasing rate outpacing the translation of research findings into practice. Coupled with the proliferation of electronic health records, and consumer health information, researchers and practitioners are challenged to leverage the full potential of EBM. In this paper we present a research agenda for leveraging business intelligence and big data analytics in evidence based medicine, and illustrate how analytics can be used to support EBM. <s> BIB004 </s> A Review on Business Intelligence and Big Data <s> Discussion <s> Big data is an opportunity in the emergence of novel business applications such as “Big Data Analytics” (BDA). However, these data with non-traditional volumes create a real problem given the capacity constraints of traditional systems. The aim of this paper is to deal with the impact of big data in a decision-support environment and more particularly in the data integration phase. In this context, we developed a platform, called P-ETL (Parallel-ETL) for extracting (E), transforming (T) and loading (L) very large data in a data warehouse (DW). To cope with very large data, ETL processes under our P-ETL platform run on a cluster of computers in parallel way with MapReduce paradigm. The conducted experiment shows mainly that increasing tasks dealing with large data speeds-up the ETL process. <s> BIB005 </s> A Review on Business Intelligence and Big Data <s> Discussion <s> Extract-Transform-Load (ETL) programs process data into data warehouses (DWs). Rapidly growing data volumes demand systems that scale out. Recently, much attention has been given to MapReduce for parallel handling of massive data sets in cloud environments. Hive is the most widely used RDBMS-like system for DWs on MapReduce and provides scalable analytics. It is, however, challenging to do proper dimensional ETL processing with Hive; e.g., the concept of slowly changing dimensions (SCDs) is not supported (and due to lacking support for UPDATEs, SCDs are complex to handle manually). Also the powerful Pig platform for data processing on MapReduce does not support such dimensional ETL processing. To remedy this, we present the ETL framework CloudETL which uses Hadoop to parallelize ETL execution and to process data into Hive. The user defines the ETL process by means of high-level constructs and transformations and does not have to worry about technical MapReduce details. CloudETL supports different dimensional concepts such as star schemas and SCDs. We present how CloudETL works and uses different performance optimizations including a purpose-specific data placement policy to co-locate data. Further, we present a performance study and compare with other cloud-enabled systems. The results show that CloudETL scales very well and outperforms the dimensional ETL capabilities of Hive both with respect to performance and programmer productivity. For example, Hive uses 3.9 times as long to load an SCD in an experiment and needs 112 statements while CloudETL only needs 4. <s> BIB006
Both big data and BI area buzz words of nowadays. Two of them have some in common. BI has positioned itself at the core of datadriven decision support system. It also has tools like reports, dashboards, and tables to emit information those who need them, hence improve decision making, increase revenue and accomplish many other organizational goals. Big data offers new opportunities and challenges for not only a bunch of sciences and disciplines but also BI. BI had been using traditional technologies before the dawn of big data. Once the big data came out, BI began to exploit its offerings, like many others. With the help of big data, BI forms more attractive and largely relevant research topic  like BI and analytics, BDA, advanced analytics. Big data enhances BI capabilities by offering new technologies and tools. BI and big data break new ground as "business intelligence and big data analytics" BIB004   . To examine big data effects on BI, typical BI architecture is used which is a cyclical set of activities from raw data to valuable and consumer-ready information BIB001 . Stages of BI architecture are as follows: Data sources, data movement (ETL), data warehousing, mid-tier servers, and front-end applications. • Data sources: Although traditional BI is able to use unstructured and semistructured data as a source, it has some limitations such as volume and speed. Before big data, data generating sources were not as much as now so, most of the data sources were composed of Online Transaction Processing (OLTP) which is based on RDBMS, like Customer Relationship Management -CRM, Enterprise Resource Planning System -ERP and any kind of enterprise information systems. But with big data, the composition of data sources and architecture of databases have changed due to unstructured and semi-structured data increase along with volume, velocity and variety change. NoSQL and NewSQL databases are gradually replacing or completing relational databases. Among the BI data sources, big databases and unstructured data sources are gaining weight. Thus, modern BI solutions need big data acquisition and storage capabilities which are two important big data value chain phases. • Data Movement (ETL): This stage integrates, cleanses and standardizes the data by ETL tools. By the big data, this stage becomes more challenging due to data source diversity and data volume, velocity, and complexity. ETL is not an easystage in classical technology, now in big data realm, it becomes more complex. To overcome such a difficulty requires novel approaches and paradigms BIB005 . New ETL tools are expected to process high volume, velocity, and variety data as fast as traditional ones. On the other hand, traditional RDBMS depended BI tools including ETL and DW solutions are in use. So new approaches for ETL should consider both RDBMS and big data challenges. Some of such novel big data ETL approaches are ETLMR BIB002  , and P-ETL BIB005 . ETL tools also need to mature to meet today's business expectations and have more user-friendly ETL tools which can handle big data as well as traditional one. • Data Warehousing: The main goal of DW is to establish single truth for the organization. In traditional BI, the raw data is loaded into new databases called data warehouse after ETL process. The change and new records in data sources are captured and reflected DW for an update. Mid-tier servers like OLAP uses DW as source databases because it offers cleaner and ready to consume highquality data than raw data sources. In this stage, traditional relational databases used to play an important role but now they are insufficient due to scalability and efficiency. So, at this stage,big data tools are getting prevalent like Hive or hybrid data warehouses. Hive is popular SQL like MapReduce driven big data DW system. Users use SQL abilities and Hive converts these queries into MapReduce job for data manipulation. However, Hive lacks some of the traditional DW capabilities like slowly changing dimensions. A dimensional RDBMS DW offers slowly changing dimensions and updates, but Hive doesn't. Nevertheless, new frameworks like CloudETL BIB006 and ETLMR enables slowly changing dimensions ETL for dimensional DW. Datalake is new DW like the term and has popped up by big data advent. Apart from DW or data marts, the data in a datalake is in its natural, uncultivated state mostly in big data repository like Hadoop. The data in datalake is accessed whenever needed. • Mid-tier Servers: This stage is consummate of data warehousing and adds more value to data on the route of the data-information-value journey. Mid-tier servers include OLAP server, search engines, data mining and analytics servers, and reporting servers. The most famous tool of this stage is OLAP, explained above. OLAP is an amazing tool for multidimensional data exploration. But big data has not yet matured as much as RDBMS in terms of integration of multidimensional data BIB003 . In this tier, more works are needed for scalable OLAP capabilities on large datasets. Traditional BI mid-tier stage is focused on mainly descriptive information, answering "what was happened?" By big data predictive information answering "what will happen?" has gained more importance. Furthermore, prescriptive information answering "what will happen and what possible responses could be and what kind of precautions could be taken?" is what businessmen demand most because this one contributes more decision-making. In terms of information currency, big data descriptive analytics refresh time have reduced in few seconds while traditional BI descriptive analytics is daily, at best a few hours. • Front-end Applications: At this stage business users consume information and then they make decisions, take actions or modify their processes. Consumption takes place via some applications like dashboards, spreadsheets, key performance indicators, ad-hoc query interfaces, inter-active data-mining tools, data exploration tools etc. Visualization is a key factor for not only classical BI but also big data impacted BI. Visualization allows users to understand information quickly which derived large and complex datasets without making a great effort. Beyond 3V define of big data further Vs are asserted and one of them visibility stressing visualization of data. • Workforce Skills: Both BI and big data requires business understanding as well as technical skills. While traditional BI skills heap up theuse of commercial software like SAP, SAS and Microsoft and BI/DW theories; big data skills are more technical and code based like R, Python, and Scala. On that note; new BI workforce will require following skills; proficiency in BI software use, business understanding and field information, coding, technical, statistical data mining and machine learning skills.