As long as its physical mode, yes doing what you mention is fine.  Many customers (including some of my own) use this method as a way to prove to virtualization-hostile vendors (Oracle, MS) that problems are not VMware-related - they just slide the device(s) over to a physical host, prove that the problem still exists and demand the vendor continue troubleshooting.
Hello,   Wanted to know if there are any specific restrictions about moving a RDM (physical compatibility mode) to another physical machine?   Say I have a LUN assigned to a Windows VM as RDM. Can I assign the same LUN to a another physical Windows host?       Thanks,    Rupak 
Moving RDM to physical host