Efficient Per-Shot Convex Hull Prediction By Recurrent Learning
Adaptive video streaming relies on the construction of efficient bitrate
ladders to deliver the best possible visual quality to viewers under bandwidth
constraints. The traditional method of content dependent bitrate ladder
selection requires a video shot to be pre-encoded with multiple encoding
parameters to find the optimal operating points given by the convex hull of the
resulting rate-quality curves. However, this pre-encoding step is equivalent to
an exhaustive search process over the space of possible encoding parameters,
which causes significant overhead in terms of both computation and time
expenditure. To reduce this overhead, we propose a deep learning based method
of content aware convex hull prediction. We employ a recurrent convolutional
network (RCN) to implicitly analyze the spatiotemporal complexity of video
shots in order to predict their convex hulls. A two-step transfer learning
scheme is adopted to train our proposed RCN-Hull model, which ensures
sufficient content diversity to analyze scene complexity, while also making it
possible capture the scene statistics of pristine source videos. Our
experimental results reveal that our proposed model yields better
approximations of the optimal convex hulls, and offers competitive time savings
as compared to existing approaches. On average, the pre-encoding time was
reduced by 58.0% by our method, while the average Bjontegaard delta bitrate
(BD-rate) of the predicted convex hulls against ground truth was 0.08%, while
the mean absolute deviation of the BD-rate distribution was 0.44%