SymFormer: End-to-end symbolic regression using transformer-based architecture
Many real-world problems can be naturally described by mathematical formulas.
The task of finding formulas from a set of observed inputs and outputs is
called symbolic regression. Recently, neural networks have been applied to
symbolic regression, among which the transformer-based ones seem to be the most
promising. After training the transformer on a large number of formulas (in the
order of days), the actual inference, i.e., finding a formula for new, unseen
data, is very fast (in the order of seconds). This is considerably faster than
state-of-the-art evolutionary methods. The main drawback of transformers is
that they generate formulas without numerical constants, which have to be
optimized separately, so yielding suboptimal results. We propose a
transformer-based approach called SymFormer, which predicts the formula by
outputting the individual symbols and the corresponding constants
simultaneously. This leads to better performance in terms of fitting the
available data. In addition, the constants provided by SymFormer serve as a
good starting point for subsequent tuning via gradient descent to further
improve the performance. We show on a set of benchmarks that SymFormer
outperforms two state-of-the-art methods while having faster inference.