Upsampling and Depth Estimation.    The depth map at iteration \(t\)  is estimated by sampling the depth hypotheses via linear interpolation given the index field \(\phi _t\) . Since \(\phi _t\)  is at 1/4 resolution, we upsample it to full resolution using a convex combination of a 3\(\times \) 3 neighbors as in [1]}. Specifically, a weight mask \(W_0 \in \mathbb {R} ^ {H/4 \times W/4 \times (4 \times 4 \times 9) }\)  is predicted from the hidden state \(h_t\)  using two convolutional layers and \(\operatorname{softmax}\)  is performed over the weights of those 9 neighbors. The final high resolution index field \(\phi _t^{u}\)  is obtained by taking a weighted combination over the 9 neighbors, and reshaping to the resolution H\(\times \) W. Convex combination can be implemented using the einsum function in PyTorch.
