To investigate the effectiveness of each component in MVT, we conduct detailed ablation studies on Nr3D [1]}, shown in Tab. REF .
The baseline model in row (a) simply passes object features through several linear layers and then fuses object features with language features through multiplication directly. Thereby the baseline model does not model the relationship between objects. After we introduce the transformer decoder [2]} to model object relationship and vision-language fusion in row (b), the performance is improved from \(36.9\%\)  to \(40.4\%\) .
We add random rotation augmentation \(R(\theta _{aug})\)  to the input scene in row (c), where \(\theta _{aug}\)  is randomly sampled from \(\theta _{v}\)  in Eq.(REF ) with \(N=4\) .
The improvement in the “Easy” split shows that the augmentation improves the performance of the object encoder. However, the rotation augmentation cannot eliminate the dependence on the specific views, and the performance in the “View-dep” split degraded from \(38.4\%\)  to \(35.2\%\) .
Row (d) and (e) bear out the effectiveness of the proposed language-guided supervision and multi-view modeling, respectively. They improve the performance to \(46.2\%\)  and \(52.3\%\)  respectively. The improvement of language-guided supervision comes from better vision-language alignments. The giant jump of the “View-dep.” split from \(35.2\%\)  to \(49.1\%\)  shows that our proposed multi-view space modeling effectively learns a view-robust representation, eliminating the dependence on specific views. Adopting both language-guided supervision and multi-view encoding in (g) can improve the overall accuracy to \(55.1\%\) , surpassing all previous methods.
We also compare the performance w/ and w/o rotation augmentation after adding the multi-view modeling. Comparing the performance in row (f) and row (g), we find that after adding the multi-view modeling, the rotation augmentation can enhance the object encoder and cause no degradation in the “View-dep.” split.
<FIGURE><TABLE>