STREAMLINE: A Simple, Transparent, End-To-End Automated Machine Learning Pipeline Facilitating Data Analysis and Algorithm Comparison
Machine learning (ML) offers powerful methods for detecting and modeling
associations often in data with large feature spaces and complex associations.
Many useful tools/packages (e.g. scikit-learn) have been developed to make the
various elements of data handling, processing, modeling, and interpretation
accessible. However, it is not trivial for most investigators to assemble these
elements into a rigorous, replicatable, unbiased, and effective data analysis
pipeline. Automated machine learning (AutoML) seeks to address these issues by
simplifying the process of ML analysis for all. Here, we introduce STREAMLINE,
a simple, transparent, end-to-end AutoML pipeline designed as a framework to
easily conduct rigorous ML modeling and analysis (limited initially to binary
classification). STREAMLINE is specifically designed to compare performance
between datasets, ML algorithms, and other AutoML tools. It is unique among
other autoML tools by offering a fully transparent and consistent baseline of
comparison using a carefully designed series of pipeline elements including:
(1) exploratory analysis, (2) basic data cleaning, (3) cross validation
partitioning, (4) data scaling and imputation, (5) filter-based feature
importance estimation, (6) collective feature selection, (7) ML modeling with
`Optuna' hyperparameter optimization across 15 established algorithms
(including less well-known Genetic Programming and rule-based ML), (8)
evaluation across 16 classification metrics, (9) model feature importance
estimation, (10) statistical significance comparisons, and (11) automatically
exporting all results, plots, a PDF summary report, and models that can be
easily applied to replication data.