@inproceedings{
takeda2023multimodal,
title={Multi-modal Foundation Model for Material Design},
author={Seiji Takeda and Indra Priyadarsini and Akihiro Kishimoto and Hajime Shinohara and Lisa Hamada and Hirose Masataka and Junta Fuchiwaki and Daiju Nakano},
booktitle={AI for Accelerated Materials Design - NeurIPS 2023 Workshop},
year={2023},
url={https://openreview.net/forum?id=EiT2bLsfM9}
}
We propose a multi-modal foundation model for small molecules, a shift from traditional AI models that are tailored for individual tasks and modalities. This model uses a late fusion strategy to align and fuse three distinct modalities: SELFIES, DFT properties, and optical spectrum. The model is pre-trained with over 6 billion samples to provide two primary functions, generating fused feature representations across the three modalities, and cross-modal predictions and genrations. As preliminary experiments, we demonstrate that the fused representation successfully improves the performance of property predictions for chromophore molecules, and showcase 6 distinct cross-modal inferences.
Multi-modal Foundation Model for Material Design
Seiji Takeda Indra Priyadarsini Akihiro Kishimoto Hajime Shinohara Lisa Hamada Hirose Masataka Junta Fuchiwaki Daiju Nakano