Zeroth-Order SciML: Non-intrusive Integration of Scientific Software with Deep Learning
Using deep learning (DL) to accelerate and/or improve scientific workflows
can yield discoveries that are otherwise impossible. Unfortunately, DL models
have yielded limited success in complex scientific domains due to large data
requirements. In this work, we propose to overcome this issue by integrating
the abundance of scientific knowledge sources (SKS) with the DL training
process. Existing knowledge integration approaches are limited to using
differentiable knowledge source to be compatible with first-order DL training
paradigm. In contrast, our proposed approach treats knowledge source as a
black-box in turn allowing to integrate virtually any knowledge source. To
enable an end-to-end training of SKS-coupled-DL, we propose to use zeroth-order
optimization (ZOO) based gradient-free training schemes, which is
non-intrusive, i.e., does not require making any changes to the SKS. We
evaluate the performance of our ZOO training scheme on two real-world material
science applications. We show that proposed scheme is able to effectively
integrate scientific knowledge with DL training and is able to outperform
purely data-driven model for data-limited scientific applications. We also
discuss some limitations of the proposed method and mention potentially
worthwhile future directions.