Instead of directly regressing SMPL parameters, we train a neural network to predict the coordinate heatmaps in three dimensions for each human joint and mesh vertex.
The dense heatmap-based representation can preserve the spatial relationship in the image domain and model the uncertainty of predictions.
It is shown to be effective in localizing visible joints/vertices and flexible to fit an input image faithfully [1]}, [2]}, [3]}, [4]}.
Nonetheless, the x and y-axis heatmaps are defined in the image coordinates, which cannot represent the out-of-frame (i.e., truncated by image boundaries) body parts.
In addition, occlusions by objects or the human body itself could cause ambiguity for depth-axis predictions.
Without knowing which joints/vertices are visible, the network tends to produce erroneous outputs on partial-body images.
To address this, we propose Visibility-aware Dense Body (VisDB), a heatmap-based dense representation augmented by visibility.
Specifically, we train a network to predict binary truncation and occlusion labels along with the heatmaps for each human joint and vertex.
With the visibility modeling, the proposed network can learn to make more accurate predictions based on the observable cues.
In addition, the vertex-level occlusion predictions can serve as a depth ordering signal to constrain depth predictions.
Finally, by using visibility as the confidence of 3D mesh prediction, we demonstrate that VisDB is a powerful intermediate representation which allows us to regress and/or optimize SMPL parameters more effectively.
In Figure REF , we show examples of truncation and occlusions as well as the dense human body estimations with and without visibility modeling.
