@inproceedings{
ilbert2024analysing,
title={Analysing Multi-Task Regression via Random Matrix Theory with Application to Time Series Forecasting},
author={Romain Ilbert and Malik Tiomoko and Cosme Louart and Ambroise Odonnat and Vasilii Feofanov and Themis Palpanas and Ievgen Redko},
booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
year={2024},
url={https://openreview.net/forum?id=FFW6rPz48Z}
}
In this paper, we introduce a novel theoretical framework for multi-task regression, applying random matrix theory to provide precise performance estimations, under high-dimensional, non-Gaussian data distributions. We formulate a multi-task optimization problem as a regularization technique to enable single-task models to leverage multi-task learning information. We derive a closed-form solution for multi-task optimization in the context of linear models. Our analysis provides valuable insights by linking the multi-task learning performance to various model statistics such as raw data covariances, signal-generating hyperplanes, noise levels, as well as the size and number of datasets. We finally propose a consistent estimation of training and testing errors, thereby offering a robust foundation for hyperparameter optimization in multi-task regression scenarios. Experimental validations on both synthetic and real-world datasets in regression and multivariate time series forecasting demonstrate improvements on univariate models, incorporating our method into the training loss and thus leveraging multivariate information.
Analysing Multi-Task Regression via Random Matrix Theory with Application to Time Series Forecasting
Romain Ilbert Malik Tiomoko Cosme Louart Ambroise Odonnat Vasilii Feofanov Themis Palpanas Ievgen Redko