Orbits of $GL(m,k)\times GL(n,k)$ on the set $M(m\times n, k)$ I am interested in the orbits of $GL(m,k)\times GL(n,k)$ on the set $M(m\times n, k)$, where $(g_1,g_2)\circ x:=g_1 xg_2^{-1}$.

I know that left-multiplication with a regular matrix is basically a row-transformation and right-multiplication a column-transformation. So in total the rank stays the same under this group operation. 
But is it true that for matrices $x\neq y$ of the same rank there always exist column- and row-transformations such that $g_1 xg_2^{-1}=y$?
Here is a direct solution using the Singular Value Decomposition (SVD)

(see for example (https://davetang.org/file/Singular_Value_Decomposition_Tutorial.pdf))

The principle of this decomposition is that every $m \times n$ matrix $M$ can be written in the following way:

$$\tag{1}M = U \Sigma V^{-1} \ \ \ \ \ (\text{=} \ U \Sigma V^{T}) $$

where 


$U \in O(m,\mathbb{R}) \subset GL(m,\mathbb{R})$,  
$V \in O(n,\mathbb{R}) \subset GL(n,\mathbb{R})$,  and
$\Sigma$ is $m \times n$, with "diagonal elements":


$$\Sigma_{11} \geq \Sigma_{22} \geq \cdots \Sigma_p \geq 0 \ \ \text{where} \ \  p=\min(m,n),$$

all other values $\Sigma_{ij}$ for $i \neq j$ being zero (a kind of generalized diagonal matrix). Here is an example with $m=4$ and $n=3$:

$$\underbrace{\begin{pmatrix}7   &amp;  1  &amp;   1\\
    1  &amp;  -1 &amp;  -7\\
    7 &amp;   -1 &amp;    1\\
    1  &amp;   1  &amp; -7\end{pmatrix}}_{M}=
\underbrace{\begin{pmatrix}   -0.7  &amp;  0.1 &amp;  0.5  &amp; -0.5\\
   -0.1  &amp; -0.7 &amp;  -0.5  &amp; -0.5\\
   -0.7  &amp;  0.1  &amp; -0.5  &amp;  0.5\\
   -0.1  &amp; -0.7  &amp; 0.5  &amp;  0.5\end{pmatrix}}_{U}
\underbrace{\begin{pmatrix} 10 &amp;         0     &amp;    0 \\
         0  &amp; 10 &amp;  0\\
         0  &amp;    0 &amp; 2\\
         0   &amp;    0  &amp;     0\end{pmatrix}}_{\Sigma}\underbrace{\begin{pmatrix}    -1  &amp;   0  &amp;   0\\
     0  &amp;   0   &amp;  1\\
     0  &amp;   1   &amp;  0\end{pmatrix}}_{V^{-1}}$$

Let consider now the particular case $m=3$ and $n=2$.

The orbits can thus be ''represented'' by matrices of the form:

$$\Sigma=\begin{pmatrix}a&amp;0\\0&amp;b\\0&amp;0\end{pmatrix}$$

but in fact, if $a$ and/or  $b$ are nonzero coefficients, they can be "incorporated" into matrix $U$ ; thus, the "representatives" can be taken as:

$$\Sigma_0=\begin{pmatrix}0&amp;0\\0&amp;0\\0&amp;0\end{pmatrix}, \ \ \Sigma_1=\begin{pmatrix}1&amp;0\\0&amp;0\\0&amp;0\end{pmatrix}, \ \ 
\Sigma_2=\begin{pmatrix}1&amp;0\\0&amp;1\\0&amp;0\end{pmatrix}$$

(resp. null matrix, rank 1 matrices, and rank 2 matrices).

From here, the general case is straightforward.
Here is a direct solution using the Singular Value Decomposition (SVD)

(see for example (https://davetang.org/file/Singular_Value_Decomposition_Tutorial.pdf))

The principle of this decomposition is that every $m \times n$ matrix $M$ can be written in the following way:

$$\tag{1}M = U \Sigma V^{-1} \ \ \ \ \ (\text{=} \ U \Sigma V^{T}) $$

where 


$U \in O(m,\mathbb{R}) \subset GL(m,\mathbb{R})$,  
$V \in O(n,\mathbb{R}) \subset GL(n,\mathbb{R})$,  and
$\Sigma$ is $m \times n$, with "diagonal elements":


$$\Sigma_{11} \geq \Sigma_{22} \geq \cdots \Sigma_p \geq 0 \ \ \text{where} \ \  p=\min(m,n),$$

all other values $\Sigma_{ij}$ for $i \neq j$ being zero (a kind of generalized diagonal matrix). Here is an example with $m=4$ and $n=3$:

$$\underbrace{\begin{pmatrix}7   &amp;  1  &amp;   1\\
    1  &amp;  -1 &amp;  -7\\
    7 &amp;   -1 &amp;    1\\
    1  &amp;   1  &amp; -7\end{pmatrix}}_{M}=
\underbrace{\begin{pmatrix}   -0.7  &amp;  0.1 &amp;  0.5  &amp; -0.5\\
   -0.1  &amp; -0.7 &amp;  -0.5  &amp; -0.5\\
   -0.7  &amp;  0.1  &amp; -0.5  &amp;  0.5\\
   -0.1  &amp; -0.7  &amp; 0.5  &amp;  0.5\end{pmatrix}}_{U}
\underbrace{\begin{pmatrix} 10 &amp;         0     &amp;    0 \\
         0  &amp; 10 &amp;  0\\
         0  &amp;    0 &amp; 2\\
         0   &amp;    0  &amp;     0\end{pmatrix}}_{\Sigma}\underbrace{\begin{pmatrix}    -1  &amp;   0  &amp;   0\\
     0  &amp;   0   &amp;  1\\
     0  &amp;   1   &amp;  0\end{pmatrix}}_{V^{-1}}$$

Let consider now the particular case $m=3$ and $n=2$.

The orbits can thus be ''represented'' by matrices of the form:

$$\Sigma=\begin{pmatrix}a&amp;0\\0&amp;b\\0&amp;0\end{pmatrix}$$

but in fact, if $a$ and/or  $b$ are nonzero coefficients, they can be "incorporated" into matrix $U$ ; thus, the "representatives" can be taken as:

$$\Sigma_0=\begin{pmatrix}0&amp;0\\0&amp;0\\0&amp;0\end{pmatrix}, \ \ \Sigma_1=\begin{pmatrix}1&amp;0\\0&amp;0\\0&amp;0\end{pmatrix}, \ \ 
\Sigma_2=\begin{pmatrix}1&amp;0\\0&amp;1\\0&amp;0\end{pmatrix}$$

(resp. null matrix, rank 1 matrices, and rank 2 matrices).

From here, the general case is straightforward.