1. A method of decoding video data, the method comprising: searching a set of neighbor blocks of the video data to generate a plurality of disparity vector candidates, wherein each of the neighbor blocks is a neighbor of a current block of the video data; determining, based at least in part on the plurality of disparity vector candidates, a final disparity vector for the current block; and reconstructing a sample block for the current block in part by performing, based on the final disparity vector for the current block, inter-view motion prediction or inter-view residual prediction for the current block.