@inproceedings{
anonymous2023renamer,
title={Renamer: A Transformer Architecture Invariant to Variable Renaming},
author={Anonymous},
booktitle={Machine Learning for Systems 2023},
year={2023},
url={https://openreview.net/forum?id=AilSfYM8wN}
}
Many modeling tasks involve learning functions which are invariant to certain types of input transformations. We study a specific class of invariance: semantics- preserving variable renaming for models of code. We show that vanilla Transformers trained on renaming-invariant tasks do not exhibit renaming invariance. We propose Renamer, a Transformer architecture which is itself invariant to semantics- preserving variable renaming. On a CPU simulation task, Renamer reduces error by between 24.79% and 52.8% compared to a vanilla Transformer.
Renamer: A Transformer Architecture Invariant to Variable Renaming