Interactive Healthcare Big Data Analytics Platform under Simulated Performance
To utilize data from hospital systems, big data analytics (BDA) has become increasingly important. BDA enable queries of large highly diverse and real volumes of patient data in an interactively dynamic way that enriches the use of the platform with data visualization for healthcare. We established a Healthcare BDA (HBDA) platform at the University of Victoria (UVic) with Compute Canada/Westgrid, and Vancouver Island Health Authority (VIHA) Victoria, BC, Canada. The framework was a proof of concept implementation that tested emulated patient data representative of the main hospital system at VIHA. We cross referenced all data, its profiles and metadata, with the existing clinical reporting. Our HBDA platform and its performance was tested for different patient query types in simulation with the data ingested into Hadoop file system over different applications of Apache Spark with Zeppelin and Jupyter web based interfaces, and Apache Drill interfaces. The results showed that the ingestion time of one billion records took circa 2 hours via Apache Spark. Apache Drill outperformed Spark/Zeppelin and Spark/Jupyter. However, it was restricted to running more simplified queries, and very limited in its visualizations exhibiting poor usability for healthcare. Zeppelin running on Spark showed ease of use interactions for health applications, but it lacked the flexibility of its interface tools and required extra setup time before running queries. Jupyter on Spark offered high performance stacks not only over our HBDA platform but also in unison to run all queries simultaneously with high usability for a variety of reporting requirements by providers and health professionals.
2016 IEEE 14th Intl Conf on Dependable, Autonomic and Secure Computing, 14th Intl Conf on Pervasive Intelligence and Computing, 2nd Intl Conf on Big Data Intelligence and Computing and Cyber Science and Technology Congress(DASC/PiCom/DataCom/CyberSciTech)