arXiv:2201.04600cs.LG
Deep Symbolic Regression for Recurrent Sequences
Abstract
Symbolic regression, i.e. predicting a function from the observation of its values, is well-known to be a challenging task. In this paper, we train Transformers to infer the function or recurrence relation underlying sequences of integers or floats, a typical task in human IQ tests which has hardly been tackled in the machine learning literature. We evaluate our integer model on a subset of OEIS sequences, and show that it outperforms built-in Mathematica functions for recurrence prediction. We also demonstrate that our float model is able to yield informative approximations of out-of-vocabulary functions and constants, e.g. $\operatorname{bessel0}(x)\approx \frac{\sin(x)+\cos(x)}{\sqrt{πx}}$ and $1.644934\approx π^2/6$. An interactive demonstration of our models is provided at https://symbolicregression.metademolab.com.
Nothing prevents it, no. Transformers are certainly capable of learning mathematical tasks; consider [1] as an example, which uses big but regular token lengths. Alternatively you could just scale 'till the problem solves itself. [1] https://arxiv.org/abs/2201.04600
VeedracView on HNParent context