3D Visual Grounding. The task of 3D visual grounding aims to ground objects described by natural language in 3D scenes. Referit3D [1]} and ScanRefer [2]} have built benchmarks and baselines for the 3D visual grounding task with language query annotations on ScanNet [3]} dataset. Recent works [4]}, [1]}, [2]}, [7]}, [8]}, [9]} solve the 3D grounding tasks following a two-stage scheme. In the first stage, 3D object proposals are generated either with ground truth [1]} or by a 3D object detector [2]}, [12]}. In the second stage, the task is modeled as a matching problem where 3D visual features are fused with the text features of the query language description to predict the matching scores. Referit3D [1]} uses GRU [14]} to extract text features, and makes use of GNN [15]} to model the relationship between objects. With the widespread of transformers [16]}, recent works [9]}, [7]}, [8]} have explored transformers for feature extraction and feature fusion. Among them, InstanceRefer [4]} conducts multi-level contextual referring by considering attributes and local relations. LanguageRefer [8]} replaces point cloud features with predicted labels of objects, transforming the multi-modal task into a language modeling problem. SAT [9]} has conducted joint training of 2d grounding and 3d grounding, improving the performance significantly by cross-modal knowledge transfer.
3DVG-Transformer [23]} proposes a coordinate-guided contextual aggregation module and a multiplex attention module to fully utilize the contextual clues in the 3D scene.
FFL-3DOG [24]} uses language and visual scene graphs to guide the feature matching between language and point cloud.
TransRefer3D [7]} designs an entity-aware attention module and a relation-aware attention module to conduct fine-grained cross-modal feature matching.
Different from previous works, we start by studying the intrinsic problem of 3D visual data and build a view-robust multi-modal representation to solve the problem.
