Twitter will transfer some of its core infrastructures to the cloud-based Googles platform for better data management, the company announced. "We are pleased to announce that we are working with Google Cloud to move the cold data warehouse and Hadoop's flexible computing clusters to the Google Cloud platform," Parag Agrawal, chief technology officer of Twitter, said Thursday. "This will allow us to increase the experience and productivity of our engineering teams working with our data platform," he added.
Hadoop from Apache is an open source software for the organization of Big Data. The Hadoop computing system is the foundation of the Twitter data platform, and the company is launching several large Hadoop clusters, which are some of the largest in the world.
"In fact, our Hadoop file systems contain more than 300 PB of data for tens of thousands of servers," Agrawal said. The Google Cloud Platform solutions and reliable infrastructure will provide Twitter with the technical flexibility and consistency required by the platform.
Migration, once completed, will accelerate the creation of resources, increase flexibility, access to a wider ecosystem of tools and services, and improve safety.
"Architecturally, we will also be able to share the calculations and storage for this class of Hadoop workloads, which has a number of long-term scalable and operational advantages," the report said.