Online Social Network (OSN) is one of the most hottest services in the past years. It preserves the life of users and provides great potential for journalists, sociologists and business analysts. Crawling data from social network is a basic step for social network information analysis and processing. As the net- work becomes huge and information on the network updates faster than web pages, crawling is more dicult because of the limitations of band-width, po- liteness etiquette and computation power. To extract fresh information from social network eciently and eectively, this paper presents a novel crawling method and discusses parallelization architecture of social network. To dis- cover the feature of social network, we gather data from real social network, analyze them and build a model to describe the discipline of users' behavior. With the modeled behavior, we propose methods to predict users' behavior. According to the prediction, we schedule our crawler more reasonably and ex- tract more fresh information with parallelization technologies. Experimental results demonstrate that our strategies could obtain information from OSN eciently and eectively.
Only a few methods are proposed to crawl OSN data. @cite_16 describes a Twitter Crawler developed by Java. They pay more attention to the implementation detail of the crawler and the data analysis. Instead, we focus on the crawling method and develop algorithms to gather more information of the specific OSN users.
@cite_16
2017114019
Applying data mining techniques to social media can yield interesting perspectives about individual human behavior, detecting hot issues and topics, or discovering a group and community. However, it is difficult to build your own data set to apply data mining techniques without an automated data gathering system. To overcome this challenge, we developed a java-based data gathering tool that continually collects social data from Twitter. This allows us, as well as other researchers, to build our own Twitter database. In this paper, we introduce the design specifications and explain the implementation details of the Twitter Data Collecting Tool we developed. In addition, we provide an in-depth analysis of Twitter messages about various Super Bowl ads by applying data-mining techniques to a case study. The study aims to address the question of how people use Twitter and to assess the power of Twitter in terms of creating consumer interest in brands and commercials.