Therefore, towards an end-to-end, differentiable solution, we propose to use a GRU-based module to implicitly optimize the matching volume. It estimates a sequence of index fields, \(\lbrace  \phi _t \rbrace _{t=1}^{T}\)  by unrolling the optimization problem to \(T\)  iterative updates (in a descent direction), mimicking the updates of a first-order optimizer according to [1]}, [2]}, [3]}, [4]}. At each iteration \(t\) , the index field \(\phi _t \in \mathbb {R} ^{H \times W} \)  is estimated as a grid of indices to iteratively approach better (i.e., closer to the ground truth) depth hypotheses having a lower matching cost. Specifically, a residual index field \(\delta \phi _t\)  is predicted as an update direction for next iteration, i.e., \(\phi _{t+1} = \phi _t + \delta \phi _t\) , (analogous to the direction \(\mathbf {r}\)  in SGM), which is explicitly driven by training the system (e.g., feature encoders, the transformer layer, and the residual pose net, etc.) to minimize the loss between the predicted depth maps and the ground truth. The recurrent estimate of the index field enables the learning to be directly anchored at the cost volume domain.
This indexing paradigm differentiates our approach from other depth estimation methods, such as the depth regression in 2D CNNs which fuses cost volume and the skipped multi-level features [5]}, [6]}, [7]}, and 3D CNNs by soft-argmin [8]} after cost volume aggregation and regularization by 3D convolutions [9]}, [10]}, [11]}, [12]}.
