between the two objective functions \(J_1(\mathbf {p})\)  and \(J_2(\mathbf {p})\) , for different values of the parameter \(\alpha \in [0,1]\)  (an adaptive version of the method can be applied in cases for which the classical weighted sum method fails, e.g., when the Pareto frontier is nonconvex [1]}). Assuming that both \(J_1(\mathbf {p})\)  and \(J_2(\mathbf {p})\)  are differentiable and the optimization problem is unconstrained (i.e., \(P=\mathbb {R}^n\) ) or that it can be reduced to an unconstrained optimization problem by using a suitable penalization approach, one could perform the optimization numerically by applying the classical gradient method, possibly combined with a multi-start approach. In order to reduce the computational effort needed for the exact computation of the gradient at each iteration of the gradient method, one could replace it with its approximation obtained by applying PCA to the gradient field \(\nabla J_\alpha (\mathbf {p})\)  evaluated on a subset of points \(\mathbf {p}_j \in P\)  (for \(j=1,\ldots ,m\) ), then projecting the exact gradient onto the subspace generated by the average of the gradients \(\nabla J_\alpha (\mathbf {p}_j)\) , and by the first principal directions found by PCA, when this is applied to the dataset \(\lbrace \nabla J_\alpha (\mathbf {p}_j)\rbrace _{j=1}^m\) , after a pre-processing step, which makes it centeredIt is common practice to apply PCA to centered (also called de-meaned) data matrices \(\mathbf {X}^{(c)}\) , i.e., having the form \(\mathbf {X}^{(c)} \doteq \mathbf {X}-\mathbf {1}_m \bar{\mathbf {x}}^{\prime }\) , where \(\mathbf {1}_m \in \mathbf {R}^m\)  denotes a column vector made of \(m\)  ones, and \(\bar{\mathbf {x}}\in \mathbf {R}^n\)  is a column vector whose elements are the averages of the corresponding columns of \(\mathbf {X}\) . This does not change the quality of the results of the theoretical analysis, because, by linearity, the centered convex combination of two data matrices \(\mathbf {X}_1\)  and \(\mathbf {X}_2\)  is equal to the convex combination of the two respective centered data matrices \(\mathbf {X}_1^{(c)}\)  and \(\mathbf {X}_2^{(c)}\) .. Due to the structure of the objective function \(J_\alpha (\mathbf {p})\) , such dataset (represented by a data matrix \(\mathbf {X}_\alpha \) ) would be made of the convex combination (with coefficients \(\alpha \)  and \(1-\alpha \) ) of the two datasets \(\lbrace \nabla J_1(\mathbf {p}_j)\rbrace _{j=1}^m\)  and \(\lbrace \nabla J_2(\mathbf {p}_j)\rbrace _{j=1}^m\) , represented respectively by the two data matrices \(\mathbf {X}_1\)  and \(\mathbf {X}_2\) . In this context, the results of our theoretical analysis could be useful to restrict the application of the weighted sum method to a coarse grid of values \(\alpha _k\)  for \(\alpha \in [0,1]\) , from which one could infer, for other values of \(\alpha \) , the empirical variances of the projections of the (de-meaned) data matrices \(\mathbf {X}^{(c)}(\alpha )\)  onto the principal directions either selected or discarded by PCA, when PCA is applied to each such data matrix \(\mathbf {X}^{(c)}(\alpha )\) . Moreover, in view of this application to multi-objective optimization, the theoretical analysis of this work could be extended by finding upper bounds on the Jordan canonical anglesThese, loosely speaking, represent the smallest angles between corresponding elements of the orthonormal bases of two subspaces of \(\mathbb {R}^n\) , being the bases chosen to minimize such angles. For rigorous definitions, see [2]}, [3]} and the references therein.
between the subspaces found by PCA applied to the data matrices \(\mathbf {X}(\alpha )\)  generated from \(\mathbf {X}_1\)  and \(\mathbf {X}_2\)  for two different values of \(\alpha \in [0,1]\) . Such an extension could be derived by applying a variation (proved in [2]}) of the well-known Davis-Kahan theorem in matrix perturbation theory [5]}. A second extension of the analysis to the case of nonlinear versions of PCA, such as kernel PCA [6]}, seems also possible (e.g., via the kernel trick).
