To overcome these, we present a novel design, dubbed volumetric aggregation with transformers (VAT), to effectively integrate information present in all pixel-wise matching costs between query and support with transformers [1]}. VAT is an encoder-decoder architecture. To compensate for huge complexity of standard transformers [1]} that prevents from directly applying to correlation maps, for the encoder, we present volume embedding module to effectively reduce the number of tokens while injecting some convolutional inductive bias [3]} and volume transformer module based on swin transformer [4]}. We design our encoder in a pyramidal fashion to let the output from coarser level cost aggregation to guide the finer level. We then present our decoder that utilizes appearance affinity to resolve the ambiguities in the correlation map and expedite the learning process.
