Summary: Is there an efficient out-of-place GPU tensor transpose operation that scales as $O(n)$ for tensors with $n$ total elements, regardless of the rank $d$?  The naive algorithm costs $O(dn)$, since it requires $O(d)$ worth of index manipulation per entry.Here is the current implementation of high rank tensor transpose in TensorFlow:https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/kernels/transpose_functor_gpu.cu.cc#L27It does a flat 1-D loop, deconstructs the index using $O(d)$ divisions by strides, and reconstructs the transposed index.  Is there any index trickery / precomputation that would avoid the $O(d)$ cost per entry?
Efficient out-of-place arbitrary rank GPU transpose
tensor