My problem at hand pertains to choosing a distance measure for use in locally weighted regression. In my particular problem, I have a data set that is upwards of 10 dimensions, where the variables have different units (think distances, speeds, angles).The way I am formulating the problem is along the following lines:Goal is to find the optimal solution to the following:$$ M^* = \text{arg}\min \frac{1}{2N} \sum_{i=1}^N \left(f(\textbf{x}_i,M) - y_i\right)^2$$Given that:$$ \text{d}(\textbf{u},\textbf{v},M) = \sqrt{(\textbf{u}-\textbf{v})^TM^TM(\textbf{u}-\textbf{v})}$$$$ f(\textbf{x},M) = \sum_{j=1}^p\beta_j(\textbf{x},M)\phi_j(\textbf{x})$$where $\text{d}(\cdot,\cdot,\cdot)$ is the distance measure for the data set based on some weighting matrix $M$, $\mathbf{\beta}_j(\textbf{x},M)$ is the $j^{th}$ term in the local least square coefficient solution based on some weighting matrix $M$, and $\phi_j$ is the $j^{th}$ basis function used to represent the unknown function locally. Note that to find the $\mathbf{\beta}_j$, a weighted least square problem, where the weights nonlinearly depend on $M$, is used.Based on this formulation, the optimization problem to find $M^*$ is non-convex, so I would like to know if there's any other approaches that might be better? I haven't had much success thus far finding any information, though I might not know the right thing to search for.I want to note that I actually have tried scaling each term of the vector, based on the variance in its possible values, and this did not really work. I should also note for the angular measurements, I actually find the smallest signed angle difference in the $(\textbf{u}-\textbf{v})$ operation (ie $359^o-6^o=-7^o)$.
Methods to Estimate Optimal Distance Measure for Multidimensional Data Set
optimization;regression;distance measure