@inproceedings{
garau-luis2024multimodal,
title={Multi-modal Transfer Learning between Biological Foundation Models},
author={Juan Jose Garau-Luis and Patrick Philippe Bordes and Liam Gonzalez and Ma{\v{s}}a Roller and Bernardo P de Almeida and Christopher F. Blum and Lorenz Hexemer and Stefan Laurent and Maren Lang and Thomas PIERROT and Guillaume Richard},
booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
year={2024},
url={https://openreview.net/forum?id=xImeJtdUiw}
}
Biological sequences encode fundamental instructions for the building blocks of life, in the form of DNA, RNA, and proteins. Modeling these sequences is key to understand disease mechanisms and is an active research area in computational biology. Recently, Large Language Models have shown great promise in solving certain biological tasks but current approaches are limited to a single sequence modality (DNA, RNA, or protein). Key problems in genomics intrinsically involve multiple modalities, but it remains unclear how to adapt general-purpose sequence models to those cases. In this work we propose a multi-modal model that connects DNA, RNA, and proteins by leveraging information from different pre-trained modality-specific encoders. We demonstrate its capabilities by applying it to the largely unsolved problem of predicting how multiple \rna transcript isoforms originate from the same gene (i.e. same DNA sequence) and map to different transcription expression levels across various human tissues. We show that our model, dubbed IsoFormer, is able to accurately predict differential transcript expression, outperforming existing methods and leveraging the use of multiple modalities. Our framework also achieves efficient transfer knowledge from the encoders pre-training as well as in between modalities. We open-source our model, paving the way for new multi-modal gene expression approaches.
Multi-modal Transfer Learning between Biological Foundation Models
Juan Jose Garau-Luis Patrick Philippe Bordes Liam Gonzalez Maša Roller Bernardo P de Almeida Christopher F. Blum Lorenz Hexemer Stefan Laurent Maren Lang Thomas PIERROT Guillaume Richard