Limiting Degrees of Freedom in 3D Point Registration I'm search for some assistance in my application of Arun's algorithm for registration (fitting) of two 3D point sets using the Singular Value Decomposition: http://uploads.tombertalan.com/13spring2013/520apc520/materials/Closest%20Orthogonal%20Transformation%20using%20SVD/arun.pdf
The major point of this algorithm is that rotation (3 DOF) and translation (3 DOF) can be separated and the rotation solved for first by subtracting the centroid from each point set. 
Using the SVD, the ideal rotation is determined as:
R=VU'
where:
UEV'=svd(H)
and H=covariance matrix. 
The covariance matrix is a 3x3 matrix created from multiplying point set 1 by the transpose of set 2. 
My issue is how can I reduce the number of rotations determined by this method? If I only want to allow best fitting through the rotation of two axis I imagine I must create my covariance matrix differently. Perhaps someone here can shed some light on this. 
I have matlab code of my implementation but I'm not sure if it will be helpful. 

The algorithm article uses the min-square solution is maximized when $Tr(RH)$ is maximized which it is when $R$ is chosen such that $RH$ is positive definite and symmetrical. This is very similar to Orthogonal Procrustes problem. As I understand it you want to want to limit the axle of rotation to a plane. 
I see a couple of ways to do this.
If your orthogonal component is small. Calculate $R$ as in the article, find the axle of rotation which is the eigenvector $R$ corresponding to the eigenvalue $1$ and project it to that plane by zeroing corresponding component. Recalculate your rotation around that axis. This is probably not a very good method.
Find two one dimensional rotations that fit the matrix the best. 
Say that you want rotate around the third axis. The rotation matrix can then be written as $$R = \begin{bmatrix}R'&0\\0&1\end{bmatrix}$$ where $R^3$ is a $2\times2$ rotation matrix. This means that $Tr(RH)=Tr(R'H')+H_{33}$ where $H'$ is the top left quadratic submatrix of $H$. Calculate $R'$ as you would with $R$ and apply and repeat with the other axle. I guess that you have to do some theoretical work to work out how good this approximation is.
Join the dark side, ignore fancy orthogonality and such. You only need to work with two degrees of freedom anyway. Maximizing $Tr(R_y(\phi)R_z(\theta)H)$ where $R_x,R_y$ are Givens rotations is probably quite easy. 
