1. A method to determine a 3D human pose, comprising: a. learning visual words for human pose estimation through a supervised method; b. deriving a separate metric for each visual word from labeled image-to-pose pairs through supervised learning; c. representing a multi-modal distribution of the 3D human pose space conditioned on a feature space with a Bayesian mixture of experts (BME) model; and d. jointly optimizing metric learning and the BME model by an iterative gradient ascent process.