This paper aims to reduce the prebuffering requirements, while maintaining continuity, for video streaming. Current approaches do this by making use of adaptive media playout (AMP) to reduce the playout rate. However, this introduces playout distortion to the viewers and increases the viewing latency. We approach this by proposing a frame rate optimization framework that adjusts both the encoder frame generation rate and the decoder playout frame rate. Firstly, we model this problem as the joint adjustment of the encoder frame generation interval and the decoder playout frame interval. This model is used with a discontinuity penalty virtual buffer to track the accumulated difference between the receiving frame interval and the playout frame interval. We then apply Lyapunov optimization to the model to systematically derive a pair of decoupled optimization policies. We show that the occupancy of the discontinuity penalty virtual buffer is correlated to the video discontinuity and that this framework produces a very low playout distortion in addition to a significant reduction in the prebuffering requirements compared to existing approaches. Secondly, we introduced a delay constraint into the framework by using a delay accumulator virtual buffer. Simulation results show that the the delay constrained framework provides a superior tradeoff between the video quality and the delay introduced compared to the existing approach. Finally, we analyzed the impact of delayed feedback between the receiver and the sender on the optimization policies. We show that the delayed feedbacks have a minimal impact on the optimization policies.
AMP has also been integrated into the design of packet schedulers @cite_20 @cite_28 @cite_18 @cite_13 @cite_3 . These techniques tend to slowdown the playout rate for important video packets. This ensures that the more important video packets have a higher probability of meeting its deadline and thus avoid being dropped. We do not focus on packet scheduling in this paper and our proposed framework could complement any packet scheduling scheme.
@cite_18 @cite_28 @cite_3 @cite_13 @cite_20
2121827745 2106290740 2143614689 1995848834
We propose a new scheme for streaming media systems that combines adaptive media playout (AMP) with rate-distortion (R-D)optimized packet transmission. AMP, the client-controlled, adaptive modification of the media playout rate, allows us to flexibly adjust the playout deadlines of individual packets, and can therefore reduce reconstruction distortion. This added flexibility incurs a subjective cost, however. We introduce functions that assess the subjective cost of a schedule of playout rate modifications, and we show how to optimize the schedule with respect to these costs and distortion. Because the optimal playout rate schedule and the R-D optimal transmission schedule are interdependent, we solve the two problems jointly. In simulations that model a receiver-driven scenario, results for a short media clip show a more than 2 dB improvement in mean PSNR for R-D optimized transmission scheduling combined with a moderate amount of AMP, over R-D optimized transmission scheduling alone. When media is streamed over best-effort networks, media data is buffered at the client to protect against playout interruptions due to packet losses and random delays. While the likelihood of an interruption decreases as more data is buffered, the latency that is introduced increases. In this paper we show how adaptive media playout (AMP), the variation of the playout speed of media frames depending on channel conditions, allows the client to buffer less data, thus introducing less delay, for a given buffer underflow probability. We proceed by defining models for the streaming media system and the random, lossy, packet delivery channel. Our streaming system model buffers media at the client, and combats packet losses with deadline-constrained automatic repeat request (ARQ). For the channel, we define a two-state Markov model that features state-dependent packet loss probability. Using the models, we develop a Markov chain analysis to examine the tradeoff between buffer underflow probability and latency for AMP-augmented video streaming. The results of the analysis, verified with simulation experiments, indicate that AMP can greatly improve the tradeoff, allowing reduced latencies for a given buffer underflow probability. Multimedia communications often require intramedia synchronization for video data to prevent potential playout discontinuity resulting from network delay variation (jitter) while still achieving satisfactory playout throughput. In this paper, we propose a neural network (NN) based intravideo synchronization mechanism, called the intelligent video smoother (IVS), operating at the application layer of the receiving end system. The IVS is composed of an NN traffic predictor, an NN window determinator, and a window-based playout smoothing algorithm. The NN traffic predictor employs an on-line-trained back-propagation neural network (BPNN) to periodically predict the characteristics of traffic modeled by a generic interrupted Bernoulli process (IBP) over a future fixed time period. With the predicted traffic characteristics, the NN window determinator determines the corresponding optimal window by means of an off-line-trained BPNN in an effort to achieve a maximum of the playout quality (Q) value. The window-based playout smoothing algorithm then dynamically adopts various playout rates according to the window and the number of packets in the buffer. Finally, we show that via simulation results and live video scenes, compared to two other playout approaches, IVS achieves high-throughput and low-discontinuity playout under a mixture of IBP arrivals. We propose an application layer algorithm for content-aware adaptive video frame scheduling on bandwidth varying channels. The proposed approach includes an iterative algorithm which sequentially uses the rate constraint and delay constraint for video frames to arrive at a decision to drop send video frames. In addition to our base algorithm, we also propose an explicit deadline adjustment (EDA) algorithm. The main idea behind EDA relates to the sender adjusting postponing the deadline of an important frame which is estimated to miss its deadline by actively dropping one (or more) less important next frame(s) and sending the deadline adjusted postponed frame to be displayed in place of the next dropped frame(s). The rate-distortion (R-D) performance of our proposed approach and its comparison with prior art method is shown.