Find $\mathbf{Q}$ by minimizing $\operatorname{trace}(\mathbf{Q}^\top \mathbf{AQ}$) such that $\mathbf{Q}$ is orthonormal Suppose $\mathbf{Q} \in \mathbb{R}^{m \times k}$ where $k < m$ and $\mathbf{Q}^\top\mathbf{Q} = \mathbf{I}$.
How can I find $\mathbf{Q}$ such that
$$
\operatorname{trace}(\mathbf{Q}^\top \mathbf{AQ})
$$
is minimized where $\mathbf{A}$ is positive semi-definite?
I tried the following simplification:
$$
\mathbf{Q}^\top \mathbf{AQ} = \mathbf{Q}^\top \mathbf{UD}\mathbf{U}^\top \mathbf{Q} = \mathbf{Q}^{*\top} \mathbf{DQ}^*
$$
where $\mathbf{Q}^* = \mathbf{U}^\top\mathbf{Q}$, $\mathbf{A} = \mathbf{UDU}^\top$ and $\mathbf{Q}^{*\top}\mathbf{Q}^* = \mathbf{I}$.
If I minimize this quantity, does that mean the columns of $\mathbf{Q}^*$ is given by the eigenvectors corresponding to the smallest eigenvalues of $\mathbf{A}$?

Using your notation but replacing $Q^*$ by $V$, the problem is $$\min_{V\in \mathbb{R}^{m \times k}\\ V^{\mathrm{T}}V = I}\  trace(V^{\mathrm{T}}DV).$$
Suppose $V = (v_{ij})_{m \times k}$ and $D = diag\{\lambda_1,\cdots,\lambda_m\}$ where $0 \leq \lambda_1 \leq \cdots \leq \lambda_m$, then it is easy to coumpute that $$(V^{\mathrm{T}}DV)_{jj} = \sum_{i=1}^m \lambda_i v_{ij}^2.$$
So $$trace(V^{\mathrm{T}}DV) = \sum_{j=1}^k \sum_{i=1}^m \lambda_i v_{ij}^2 \geq \sum_{j=1}^k\lambda_j$$
And when taking $v_{jj} = 1,\ 1 \leq j \leq k$ and $v_{ij} = 0,\ i \neq  j$ we get the equality.
