Alternating conditional expectations

Alternating conditional expectations (ACE) is an algorithm to find the optimal transformations between the response variable and predictor variables in regression analysis.
Introduction.
In statistics, nonlinear transformation of variables is commonly used in practice in regression problems. Alternating conditional expectations (ACE) is one of the methods to find those transformations that produce the best fitting additive model. Knowledge of such transformations aids in the interpretation and understanding of the relationship between the response and predictors.
ACE transform the response variable formula_1 and its predictor variables, formula_2 to minimize the fraction of variance not explained. The transformation is nonlinear and is obtained from data in an iterative way.
Mathematical description.
Let formula_3 be random variables. We use formula_4 to predict formula_1. Suppose formula_6 are zero-mean functions and with these transformation functions, the fraction of variance of formula_7 not explained is
Generally, the optimal transformations that minimize the unexplained part are difficult to compute directly. As an alternative, ACE is an iterative method to calculate the optimal transformations. The procedure of ACE has the following steps:
Bivariate case.
The optimal transformation formula_19 for formula_20 satisfies
where formula_22 is Pearson correlation coefficient. formula_23 is known as the maximal correlation between formula_24 and formula_1. It can be used as a general measure of dependence.
In the bivariate case, ACE algorithm can also be regarded as a method for estimating the maximal correlation between two variables.
Software implementation.
The ACE algorithm was developed in the context of known distributions. In practice, data distributions are seldom known and the conditional expectation should be estimated from data. R language has a package acepack which implements ACE algorithm. The following example shows its usage:
 library(acepack)
 TWOPI <- 8 * atan(1)
 x <- runif(200, 0, TWOPI)
 y <- exp(sin(x) + rnorm(200)/2)
 a <- ace(x, y)
 par(mfrow=c(3,1))
 plot(a$y, a$ty) # view the response transformation
 plot(a$x, a$tx) # view the carrier transformation
 plot(a$tx, a$ty) # examine the linearity of the fitted model
Discussion.
The ACE algorithm provides a fully automated method for estimating optimal transformations in multiple regression. It also provides a method for estimating maximal correlation between random variables. Since the process of iteration usually terminates in a limited number of runs, the time complexity of the algorithm is formula_26 where formula_27 is the number of samples. The algorithm is reasonably computer efficient.
A strong advantage of the ACE procedure is the ability to incorporate variables of quite different type in terms of the set of values they can assume. The transformation functions formula_28 assume values on the real line. Their arguments can, however, assume values on any set. For example, ordered real and unordered categorical variables can be incorporated in the same regression equation. Variables of mixed type are admissible.
As a tool for data analysis, the ACE procedure provides graphical output to indicate a need for transformations as well as to guide in their choice. If a particular plot suggests a familiar functional form for a transformation, then the data can be pre-transformed using this functional form and the ACE algorithm can be rerun.
As with any regression procedure, a high degree of association between predictor variables can sometimes cause the individual transformation estimates to be highly variable, even though the complete model is reasonably stable. When this is suspected, running the algorithm on randomly selected subsets of the data, or on bootstrap samples can assist
in assessing the variability.