Energy big data: A survey <s> C. AN INTEGRATED BIG DATA ARCHITECTURE IN THE SMART GRID <s> Power utilities are increasingly rolling out “smart” grids with the ability to track consumer power usage in near real-time using smart meters that enable bidirectional communication. However, the true value of smart grids is unlocked only when the veritable explosion of data that will become available is ingested, processed, analyzed and translated into meaningful decisions. These include the ability to forecast electricity demand, respond to peak load events, and improve sustainable use of energy by consumers, and are made possible by energy informatics. Information and software system techniques for a smarter power grid include pattern mining and machine learning over complex events and integrated semantic information, distributed stream processing for low latency response, Cloud platforms for scalable operations and privacy policies to mitigate information leakage in an information rich environment. Such an informatics approach is being used in the DoE sponsored Los Angeles Smart Grid Demonstration Project, and the resulting software architecture will lead to an agile and adaptive Los Angeles Smart Grid. <s> BIB001 </s> Energy big data: A survey <s> C. AN INTEGRATED BIG DATA ARCHITECTURE IN THE SMART GRID <s> Nowadays, power systems are characterized with heavy loading conditions, abrupt changes in generation patterns, extensive switching of power system configuration and high penetration of distributed generation (DG). Under those conditions performance of the conventional relays that rely on predefined settings may deteriorate leading to system wide disturbances and blackouts. To ensure reliable system operation, monitoring, control and protection have to be improved. In this paper, new Hierarchically Coordinated Protection (HCP) approach to mitigate the effects of increased grid complexity on its operation is proposed. The proposed approach utilizes local and wide area measurements and relies on the three HCP framework levels: fault anticipation and prediction, adaptive fault detection, and relay operation correction in case of unwanted tripping. It brings intelligence to the relays at all voltage levels and uses information and statistics from the systems such as weather, lightning, animal and bird migration patterns, component outage history, etc to enhance protection system tripping dependability and security. (6 pages) <s> BIB002 </s> Energy big data: A survey <s> C. AN INTEGRATED BIG DATA ARCHITECTURE IN THE SMART GRID <s> When the amount of data is very large and it cannot be handled by the conventional database management system, then it is called big data. Big data is creating new challenges for the data analyst. There can be three forms of data, structured form, unstructured form and semi structured form. Most of the part of bigdata is in unstructured form. Unstructured data is difficult to handle. The Apache Hadoop project provides better tools and techniques to handle this huge amount of data. A Hadoop distributed file system for storage and the MapReduce techniques for processing this data can be used. In this paper, we presented our experimental work done on big data using the Hadoop distributed file system and the MapReduce. We have analyzed the variable like amount of time spend by the maps and the reduce, different memory usages by the Mappers and the reducers. We have analyzed these variables for storage and processing of the data on a Hadoop cluster. Keywords—SLOTS_MILLIS_MAPS; HDFS; MapReduce; Name Node; Secondary NameNode; Data Node; SLOTS_MILLIS_REDUCES; <s> BIB003
Based on big data analytics and cloud technology, an integrated architecture can be used in the smart grid in many ways, such as optimizing power transmission, controlling power consumption, keeping the balance between power demand and supply, etc BIB001 . This architecture takes advantages of the three technologies including big data analytics, smart grid and cloud computing, which composes an enhanced version of smart grid. The improved smart grid has the following functionalities: • Analyze the historic data and estimate the energy production. • Analyze the consumer behavior patterns about electricity consumption to estimate the demand in advance. • Keep record of the energy production from different sources and make decision to switch demands between the high/low priority. • Balance the load of the demand/supply chain. • Do efficiently on the storage/transfer of the generated power. The improved smart grid architecture is illustrated in Fig. 3 , which consists of the smart grid, the HDFS and the related cloud environment. In order to manage the storage and retrieval of massive data, HDFS is used in this system. The HDFS gives concentration on distributed storage to nodes in racks BIB003 . This architecture also contains a database including consumer behavior pattern, historic data, details in power supply and demand. Each time the system estimates the demand and calculates the supply, it will refer to the consumer behavior patterns and historic data. These data are stored in a cloud-based Cassandra DataBase (Cassandra DB). This improved smart grid utilizes a prediction algorithm BIB002 to estimate the demand and supply of electric power. In a distributed environment, the smart grid uses distributed power resources like solar, wind, nuclear sources, and is applied in many areas, such as industrial production and social infrastructure. 