Data mining Twitter for cancer, diabetes, and asthma insights
Chulis, Kimberly Ph.D. Purdue University, August 2016. Data Mining Twitter for Cancer, Diabetes, and Asthma Insights. Major Professor: Dr. Richard Feinberg Twitter may be a data resource to support healthcare research. Literature is still limited related to the potential of Twitter data as it relates to healthcare. The purpose of this study was to contrast the processes by which a large collection of unstructured disease related tweets could be converted into structured data to be further analyzed. This was done with the objective of gaining insights into the content and behavioral patterns associated with disease specific communications on Twitter. Twelve months of Twitter data related to cancer, diabetes, and asthma were collected to form a baseline dataset containing over 34 million tweets. As Twitter data in its raw form would have been difficult to manage, three separate data reduction methods were contrasted to identify a method to generate analysis files, maximizing classification precision and data retention. Each of the disease files were then run through a CHAID (chi square automatic interaction detector) analysis to demonstrate how user behavior insights vary by disease. Chi square Automatic Interaction Detector (CHAID) was a technique created by Gordon V. Kass in 1980. CHAID is a tool used to discover the relationship between variables. This study followed the standard CRISP DM data mining approach and demonstrates how the practice of mining Twitter data fits into this six stage iterative framework. The study produced