Forecasting with Big Data: A Review <s> Introduction <s> Adelaide Festival of Ideas session, The Braggs, 11:30 am, Saturday 19th October, 2013. Chaired by Susannah Eliot. <s> BIB001 </s> Forecasting with Big Data: A Review <s> Introduction <s> This paper seeks to provide an alternative forecast to that provided by the Energy Information Administration (EIA) on energy-related monthly CO2 emissions in the United States. The data on CO2 emissions from petroleum, natural gas, coal and total fossil fuels obtained via the EIA covering the period January 2005 to November 2013 is analysed and then forecasted using ARIMA, Holt-Winters, and Exponential Smoothing prior to introducing the Singular Spectrum Analysis (SSA) technique for CO2 emissions forecasting. A new combination forecast (EIA-SSA) is also introduced by merging the SSA and EIA forecasts, and is seen outperforming all models including the EIA forecast. Finally, the EIA-SSA model is used to provide an alternative 12 month ahead outlook for US energy-related CO2 emissions from December 2013 to November 2014. This research is expected to influence the methodology adopted by the EIA for forecasting CO2 emissions in the future by improving the accuracy of the forecasts, and the impact of this study will be clearer upon comparing the actual CO2 emissions in US with the EIA, and EIA-SSA forecasts over the 12 month period which follows. <s> BIB002 </s> Forecasting with Big Data: A Review <s> Introduction <s> Abstract Along with the increasing availability of large databases under the purview of National Statistical Institutes, the application of data mining techniques to official statistics is now a hot topic that is far more important at present than it was ever before. Presented in this article is a thorough review of published work to date on the application of data mining in official statistics, and on identification of the techniques that have been explored. In addition, the importance of data mining to official statistics is flagged and a summary of the challenges that have hindered its development over the course of the last two decades is presented. <s> BIB003
The Big Data phenomenon has revolutionized the modern world, and is now the hottest Data Mining topic according to polls conducted by kdnuggets.com, with the current trend expected to continue into the foreseeable future. At present there is no unified definition of Big Data, however  presented two definitions for Big Data. For academics, Big Data is "a collection of data with complexity, diversity, heterogeneity, and high potential value that are difficult to process and analyze in reasonable time", whilst for policy makers, Big Data is "a new type of strategic resource in the digital era and the key factor to drive innovation, which is changing the way of humans' current production and living" (Shi 2014, p. 6) . As Varian (2014, p. 24 ) accurately asserts, "Big data will only get bigger". This increased availability of data, which has been further escalated through the evolution of Big Data is now a major concern for a large number of industries BIB003 . It is not entirely surprising that this increasing availability of data is causing anxiety, and this is evident through the sound example presented by BIB001 where the authors state that in the modern age, we generate 70 times the information stored in the library of congress simply within the first 24 hours of a new born baby's life. In another report, it is noted that South Korea is currently upgrading its data storage capabilities associated with its national weather information system by increasing the capacity to 9.3 petabytes  and these examples provide an indication of the rate at which Big Data continues to grow. The times have truly 'changed' and we now live in an age where Big Data is identified as the leading edge for innovation, competition and productivity . For example, digital data was expected to grow from 161 exabytes in 2006 to 2837 extabytes in 2012 and is now forecasted to reach 40 trillion gigabytes in 2020  . Moreover, in the year 2008 alone the world had produced 14.7 exabytes of new data  . The emergence of Big Data is now history. What is of importance is how organizations develop the tools and means necessary for reacting to, and exploiting the increasingly available Big Data for their advantage. In line with this,  notes that there is a need for the adoption of powerful tools such as Data Mining techniques which can aid in modelling the complex relationships that are inherent in Big Data. Moreover, it is noteworthy that the recent financial crisis has seen an increase in the prolific importance of risk management in organizations, and as BIB002 states, companies are now seeking to use risk management as a tool for maximising their opportunities whilst minimising the associated threats. Herein lays the opportunity, as Big Data forecasting has the ability to improve organizational performance whilst enabling better risk management . As  states, Big Data and predictive analysis goes hand in hand in the modern age with companies focussing on obtaining real time forecasts using the increasingly available data. However, not all authors agree that Big Data is a revolutionary phenomenon.  states Big Data will be more insightful in simply connecting the dots as opposed to painting a whole new picture. For  , 2013 was the year for getting accustomed to Big Data and 2014 is the year for truly exploiting Big Data towards lucrative gains. We share and subscribe to  perception on Big Data. Accordingly, we present this review paper which aims to: provide an informative review of the forecasting techniques utilized for forecasting with Big Data; provide a concise summary of the contributions of yesteryear; and identify challenges which need be overcome as the world gears to embrace and live in the presence of Big Data. In the process, we are successful in reviewing a wide range of forecasting models which have been adopted for forecasting with Big Data. In order to enable the reader to have a clear understanding of the history, we present the review of applications differentiated by the relevant field (i.e., economics, finance, and energy among others) and topic where appropriate. Those interested in tools which can be used for manipulating Big Data are referred to Varian (2014, Table 1, p. 5) The remainder of this paper is organized as follows. In the following section we discuss the problems and potential behind Big Data forecasts whilst the associated challenges are discussed in Section 3. Section 4 provides a review of statistical and data mining techniques that have been evaluated for the purpose of forecasting with Big Data and the paper ends with some conclusions in Section 5.