This paper presents an extensible and reusable framework which addresses the problem of video quality assessment over IP networks. The proposed tool (referred to as Video-Tester) supports raw uncompressed video encoding and decoding. It also includes different video over IP transmission methods (i.e.: RTP over UDP unicast and multicast, as well as RTP over TCP). In addition, it is furnished with a rich set of offline analysis capabilities. Video-Tester analysis includes QoS and bitstream parameters estimation (i.e.: bandwidth, packet inter-arrival time, jitter and loss rate, as well as GOP size and I-frame loss rate). Our design facilitates the integration of virtually any existing video quality metric thanks to the adopted Python-based modular approach. Video-Tester currently provides PSNR, SSIM, ITU-T G.1070 video quality metric, DIV and PSNR-based MOS estimations. In order to promote its use and extension, Video-Tester is open and publicly available.
On the other hand, it is also remarkable the seminal work (EvalVid) proposed by Klaue @cite_13 , a widely used tool-set for video transmission evaluation. This tool-set does actually include transmission, some quality of service (QoS) parameter estimation (packet loss, delay and jitter) and some bitstream parameter evaluations (frame loss rate and frame jitter). EvalVid video quality evaluation techniques include PSNR, Structural Similarity index (SSIM) @cite_16 , Mean Opinion Score (MOS, mapped from PSNR) and Distortion in Inverval (DIV, mapped from MOS as described in @cite_15 ).
@cite_15 @cite_16 @cite_13
1809034209 2898677683 2132549992 2137870642 2074883210 2612410279 2748103247 2766773582 2143614689 1485536093 2165222134 1537865522 2160624927 2013131615 2576041797 1616104158 1984789278 2112656320 2089500640 2123701389
With EvalVid we present a complete framework and tool-set for evaluation of the quality of video transmitted over a real or simulated communication network. Besides measuring QoS parameters of the underlying network, like loss rates, delays, and jitter, we support also a subjective video quality evaluation of the received video based on the frame-by-frame PSNR calculation. The tool-set has a modular construction, making it possible to exchange both the network and the codec. We present here its application for MPEG-4 as example. EvalVid is targeted for researchers who want to evaluate their network designs or setups in terms of user perceived video quality. The tool-set is publicly available [11]. Convolutional neural network (CNN) based methods have recently achieved great success for image super-resolution (SR). However, most deep CNN based SR models attempt to improve distortion measures (e.g. PSNR, SSIM, IFC, VIF) while resulting in poor quantified perceptual quality (e.g. human opinion score, no-reference quality measures such as NIQE). Few works have attempted to improve the perceptual quality at the cost of performance reduction in distortion measures. A very recent study has revealed that distortion and perceptual quality are at odds with each other and there is always a trade-off between the two. Often the restoration algorithms that are superior in terms of perceptual quality, are inferior in terms of distortion measures. Our work attempts to analyze the trade-off between distortion and perceptual quality for the problem of single image SR. To this end, we use the well-known SR architecture- enhanced deep super-resolution (EDSR) network and show that it can be adapted to achieve better perceptual quality for a specific range of the distortion measure. While the original network of EDSR was trained to minimize the error defined based on per-pixel accuracy alone, we train our network using a generative adversarial network framework with EDSR as the generator module. Our proposed network, called enhanced perceptual super-resolution network (EPSR), is trained with a combination of mean squared error loss, perceptual loss, and adversarial loss. Our experiments reveal that EPSR achieves the state-of-the-art trade-off between distortion and perceptual quality while the existing methods perform well in either of these measures alone. With the increasing demand for video-based applications, the reliable prediction of video quality has increased in importance. Numerous video quality assessment methods and metrics have been proposed over the past years with varying computational complexity and accuracy. In this paper, we introduce a classification scheme for full-reference and reduced-reference media-layer objective video quality assessment methods. Our classification scheme first classifies a method according to whether natural visual characteristics or perceptual (human visual system) characteristics are considered. We further subclassify natural visual characteristics methods into methods based on natural visual statistics or natural visual features. We subclassify perceptual characteristics methods into frequency or pixel-domain methods. According to our classification scheme, we comprehensively review and compare the media-layer objective video quality models for both standard resolution and high definition video. We find that the natural visual statistics based MultiScale-Structural SIMilarity index (MS-SSIM), the natural visual feature based Video Quality Metric (VQM), and the perceptual spatio-temporal frequency-domain based MOtion-based Video Integrity Evaluation (MOVIE) index give the best performance for the LIVE Video Quality Database. Mobile wireless multimedia applications (e.g., video calls and IPTV) have gained great momentum in recent years. An important issue is to monitor predict overall audiovisual quality, instead of audio-only or video-only quality, non-intrusively for technical or commercial reasons. Previous audiovisual modeling research mainly considered application parameters (e.g., codec and send bit rate). Little attention has been paid to how network parameters, e.g., Packet Error Rate (PER) affect audiovisual quality. The aim of this paper is to explore methods to predict audiovisual quality objectively for video calls in wireless applications. The contributions of the paper are twofold. Firstly, we present subjective test results on how audio and video contribute to overall audiovisual quality and develop models to reflect this relationship. Secondly, we investigated how network parameters (e.g., PER) and application parameters, e.g., video Frame Rate (FR) affect overall audiovisual quality. We developed a regression model to predict audiovisual quality from PER and FR which can be used to monitor predict audiovisual quality non-intrusively. We also explore the possibility to predict audiovisual quality from full-reference voice and video quality metrics (i.e., PESQ and PSNR) and from combined PESQ PSNR and network application parameters. The different predication accuracy obtained from these models (accuracy from 84 to 93 ) indicates the complex attributes in audiovisual quality prediction. An extended Evalvid NS-2 platform is developed to support simulation of video calls over wireless networks. It is generally recognized that severe video distortions that are transient in space and or time have a large effect on overall perceived video quality. In order to understand this phenomena, we study the distribution of spatio-temporally local quality scores obtained from several video quality assessment (VQA) algorithms on videos suffering from compression and lossy transmission over communication channels. We propose a content adaptive spatial and temporal pooling strategy based on the observed distribution. Our method adaptively emphasizes “worst” scores along both the spatial and temporal dimensions of a video sequence and also considers the perceptual effect of large-area cohesive motion flow such as egomotion. We demonstrate the efficacy of the method by testing it using three different VQA algorithms on the LIVE Video Quality database and the EPFL-PoliMI video quality database. Evaluation of coding efficiency is traditionally modeled as a continuous rate-distortion (R-D) function, where the peak signal-to-noise ratio (PSNR) is adopted as the quality measure. Although the PSNR-versus-bitrate curve offers some useful tradeoff information between video quality and coding bit-rates, it does not take human perceptual experience into account. In this work, by following the recent image video quality assessment framework based on the just-noticeable-difference (JND) notion, we conduct a subjective test for HEVC (High Efficiency Video Codec) video to measure the QP value that lies in the boundary of perceptually lossless and lossy coded bit streams for each human subject. This is also known as the first JND point. It is observed that the statistics of the first JND points of 30 subjects follows the normal distribution for a great majority of test sequences. Finally, a machine-learning approach is proposed to predict the mean of the group-based JND distribution based on extracted video features. It is shown by experimental results that the mean JND point can be predicted accurately. A continuous evaluation of the end user’s quality-of-experience (QoE) is essential for efficient video streaming. This is crucial for networks with constrained resources that offer time-varying channel quality to its users. In hypertext transfer protocol-based video streaming, the QoE is measured by quantifying the perceptual impact of distortions caused by rate adaptation or interruptions in playback due to rebuffering events. The resulting impact on the QoE due to these distortions has been studied individually in the literature. However, the QoE is determined by an interplay of these distortions, and therefore necessitates a combined study of them. To the best of our knowledge, there is no publicly available database that studies these distortions jointly on a continuous time basis. In this paper, our contributions are twofold. First, we present a database consisting of videos at full high definition and ultrahigh definition resolutions. We consider various levels of rate adaptation and rebuffering distortions together in these videos as experienced in a typical realistic setting. A subjective evaluation of these videos is conducted on a continuous time scale. Second, we present a QoE evaluation framework comprising a learning-based model during playback and an exponential model during rebuffering. Furthermore, we perform an objective evaluation of popular video quality assessment and continuous time QoE metrics over the constructed database. The objective evaluation study demonstrates that the performance of the proposed QoE model is superior to that of the objective metrics. The database is publicly available for download at http: www.iith.ac.in lfovia downloads.html . Over-the-top mobile adaptive video streaming is invariably influenced by volatile network conditions, which can cause playback interruptions (stalling or rebuffering events) and bitrate fluctuations, thereby impairing users’ quality of experience (QoE). Video quality assessment models that can accurately predict users’ QoE under such volatile network conditions are rapidly gaining attention, since these methods could enable more efficient design of quality control protocols for media-driven services such as YouTube, Amazon, Netflix, and many others. However, the development of improved QoE prediction models requires data sets of videos afflicted with diverse stalling events that have been labeled with ground-truth subjective opinion scores. Toward this end, we have created a new mobile video quality database that we call LIVE Mobile Stall Video Database-II. Our database contains a total of 174 videos afflicted with distortions caused by 26 different stalling patterns. We describe the way we simulated the diverse stalling events to create a corpus of distorted videos, and we detail the human study we conducted to obtain continuous-time subjective scores from 54 subjects. We also present the outcomes of our comprehensive analysis of the impact of several factors that influence subjective QoE, and report the performance of existing QoE-prediction models on our data set. We are making the database (videos, subjective data, and video metadata) publicly available in order to help the advance state-of-the-art research on user-centric mobile network planning and management. The database may be accessed at http: live.ece.utexas.edu research LIVEStallStudy liveMobile.html . Multimedia communications often require intramedia synchronization for video data to prevent potential playout discontinuity resulting from network delay variation (jitter) while still achieving satisfactory playout throughput. In this paper, we propose a neural network (NN) based intravideo synchronization mechanism, called the intelligent video smoother (IVS), operating at the application layer of the receiving end system. The IVS is composed of an NN traffic predictor, an NN window determinator, and a window-based playout smoothing algorithm. The NN traffic predictor employs an on-line-trained back-propagation neural network (BPNN) to periodically predict the characteristics of traffic modeled by a generic interrupted Bernoulli process (IBP) over a future fixed time period. With the predicted traffic characteristics, the NN window determinator determines the corresponding optimal window by means of an off-line-trained BPNN in an effort to achieve a maximum of the playout quality (Q) value. The window-based playout smoothing algorithm then dynamically adopts various playout rates according to the window and the number of packets in the buffer. Finally, we show that via simulation results and live video scenes, compared to two other playout approaches, IVS achieves high-throughput and low-discontinuity playout under a mixture of IBP arrivals. A major fraction of multimedia stream contents tends to be redundant and leads to wastage of storage capacity and channel bandwidth. In order to eliminate surplus data, standard video compression algorithms exploit spatial and temporal correlation present in video sequence. However, in case of a multisensor network, intersensor statistical redundancy is the most significant factor in acquiring efficient link utilization as well as making resultant findings valuable to the end user. In this paper, an extension to our previously proposed scheme has been presented to accomplish performance goals of a multisensor environment. Standard MPEG codec has been used to accomplish distributed motion compensation in prespecified directions known as directional correlation. Video frame correlation has been estimated locally at the camera node as well as across different nodes, defined as node communication strategies. Further, receiver feedback assists in quality control after reconstitution by decoder assessment. Results estimated have been analyzed for saving ratios and multimedia quality. Results analysis illustrates increased gains in frame quality and compression saving, achieved through reducing node displacement from the reference node (NR). There has recently been a great deal of interest in the development of algorithms that objectively measure the integrity of video signals. Since video signals are being delivered to human end users in an increasingly wide array of applications and products, it is important that automatic methods of video quality assessment (VQA) be available that can assist in controlling the quality of video being delivered to this critical audience. Naturally, the quality of motion representation in videos plays an important role in the perception of video quality, yet existing VQA algorithms make little direct use of motion information, thus limiting their effectiveness. We seek to ameliorate this by developing a general, spatio-spectrally localized multiscale framework for evaluating dynamic video fidelity that integrates both spatial and temporal (and spatio-temporal) aspects of distortion assessment. Video quality is evaluated not only in space and time, but also in space-time, by evaluating motion quality along computed motion trajectories. Using this framework, we develop a full reference VQA algorithm for which we coin the term the MOtion-based Video Integrity Evaluation index, or MOVIE index. It is found that the MOVIE index delivers VQA scores that correlate quite closely with human subjective judgment, using the Video Quality Expert Group (VQEG) FRTV Phase 1 database as a test bed. Indeed, the MOVIE index is found to be quite competitive with, and even outperform, algorithms developed and submitted to the VQEG FRTV Phase 1 study, as well as more recent VQA algorithms tested on this database. We develop an efficient general-purpose blind no-reference image quality assessment (IQA) algorithm that utilizes curvelet domain features of phase congruency values and local spectral entropy values in distorted images. A 2-stage framework of distortion classification followed by quality assessment is used for mapping feature vectors to prediction scores. We utilize a support vector machine (SVM) to train an image distortion and quality prediction model. The resulting algorithm which we name Phase Congruency and Spectral Entropy based Quality (PCSEQ) index is capable of assessing the quality of distorted images across multiple distortion categories. We explain the advantages of phase congruency features and spectral entropy features. We also thoroughly test the algorithm on the LIVE IQA databse and find that PCSEQ correlates well with human judgments of quality. It is superior to the full-reference (FR) IQA algorithm SSIM and several top-performance no-reference (NR) IQA methods such as DIIVINE and SSEQ. We also tested PCSEQ on the TID2008 database to ascertain whether it has performance that is database independent. We study the optimal streaming of variable bit-rate (VBR) video over a random VBR channel. The goal of a streaming application is to enable the successful decoding of each video object before its displaying deadline is violated. Hence, we define the main performance metric of a streaming system as the probability of un-interrupted video presentation, or jitter-free probability. Previous literature has described solutions to estimate the jitter-free probability by assuming either independence in the encoded data process or simplistic channel models. In this work, we present a novel analytical framework, which requires only some basic statistical information of an arbitrary VBR channel, to bound the probability of jitter-free playout under the constraint of initial playout delay and receiver buffer size. Both the infinite and finite buffer cases are considered. This technique is then applied to investigate streaming over a wireless system modeled by an extended Gilbert channel with ARQ transmission control. Experimental results with MPEG-4 VBR encoded video demonstrates that the proposed analysis derives close bounds to the actual system performance. Finally, we show that the proposed analysis provides a theoretical foundation to quantify the tradeoffs between the initial playout delay, the receiver buffer size, and the jitter-free probability for a general class of VBR streaming over random VBR channels. Newly developed hypertext transfer protocol (HTTP)-based video streaming technologies enable flexible rate-adaptation under varying channel conditions. Accurately predicting the users' quality of experience (QoE) for rate-adaptive HTTP video streams is thus critical to achieve efficiency. An important aspect of understanding and modeling QoE is predicting the up-to-the-moment subjective quality of a video as it is played, which is difficult due to hysteresis effects and nonlinearities in human behavioral responses. This paper presents a Hammerstein-Wiener model for predicting the time-varying subjective quality (TVSQ) of rate-adaptive videos. To collect data for model parameterization and validation, a database of longer duration videos with time-varying distortions was built and the TVSQs of the videos were measured in a large-scale subjective study. The proposed method is able to reliably predict the TVSQ of rate adaptive videos. Since the Hammerstein-Wiener model has a very simple structure, the proposed method is suitable for online TVSQ prediction in HTTP-based streaming. In this paper, we design a novel multiview video encoding decoding architecture for wirelessly multiview video streaming applications, e.g., 360 degrees video, Internet of Things (IoT) multimedia sensing, among others, based on distributed video coding and compressed sensing principles. Specifically, we focus on joint decoding of independently encoded compressively sampled multiview video streams. We first propose a novel side-information (SI) generation method based on a new interview motion compensation algorithm for multiview video joint reconstruction at the decoder end. Then, we propose a technique to fuse the received measurements with resampled measurements from the generated SI to perform the final recovery. Based on the proposed joint reconstruction method, we also derive a blind video quality estimation technique that can be used to adapt online the video encoding rate at the sensors to guarantee desired quality levels in multiview video streaming. Extensive simulation results of real multiview video traces show the effectiveness of the proposed fusion reconstruction method with the assistance of SI generated by an interview motion compensation method. Moreover, they also illustrate that the blind quality estimation algorithm can accurately estimate the reconstruction quality. End-to-end packet delay is an important performance parameter on the Internet, because it heavily affects the quality of real-time applications. Currently, however, because the packet transmission quality (e.g., transmission delay, jitter, packet loss) may vary dynamically, it is not easy to handle real-time traffic. For UDP-based real-time applications, a smoothing buffer (playout buffer) is typically used at the client to compensate for variable delays. The issue of playout control has been studied previously, and several algorithms for controlling the playout buffer have been proposed. These studies considered the network parameters (e.g., packet loss ratio and playout delay), but not the quality perceived by end users. We first clarify the relations between mean opinion score (MOS) of played audio and the network parameters (e.g., packet loss, packet transmission delay, and transmission rate). Then, utilizing the MOS function, we propose a new playout buffer algorithm that considers the user's perceived quality for real-time applications. Our simulation and implementation tests show that the algorithm can enhance the perceived quality more effectively than existing algorithms. In this paper, we propose a method for automatic quality of experience (QoE) prediction in stereoscopic videos. QoE, though embodying the subjective measures of the end user's perceived quality, can be expressed in relation to some quality of service (QoS) parameters. Having information on content types in modelling QoE-QoS interactions is advantageous as videos with the same QoS parameters may have different subjective scores due to different content types. Consequently, using content clustering with the help of spatio-temporal activities within depth layers, QoE predictor is designed per each content cluster utilising full reference (FR) and no reference (NR) metrics. Finally, the performance of the proposed QoE prediction algorithm is evaluated extensively and the overall measure of success value equal to 95.4 is achieved for the test sequences. This model can be applied for QoE control in video provisioning systems. This paper considers the use of the image quality metric for still image compression systems comparison. Considering the fact that the conventional PSNR cannot sufficiently reflect the result of subjective assessment other quality measures have been considered to design the variable bit-rate coders. Distortion measures are developed for the purpose of predicting the subjective quality of pictures. Subjectively relevant distortion measures that mirror viewers' assessments of picture quality would make the task of designing and optimizing coding schemes considerably easier. One significant problem associated with these measures is that there is little information on how these metrics perform in comparison to each other. The purpose of this paper is to provide a rigorous evaluation of three metrics to assess the quality of compressed images. The compression system used in this evaluation is the classical JPEG coder. Both objective and subjective tests were performed on a 50 natural image data base with a panel of experimented and non-experimented observers. The results show that these metrics are highly correlated with the subjective quality grading but also depends on the complexity of the images under study. They can be improved by integrating all masking effects and typical artifacts of the compression method. Their combination as a multidimensional quality measure in which each dimension is related to a properly identified artifact, should be based on a subjective evaluation. This paper proposes a no-reference quality assessment metric for images subject to quantization noise in block-based DCT (discrete cosine transform) domain, as those resulting from JPEG or MPEG encoding. The proposed method is based on natural scene statistics of the DCT coefficients, whose distribution is usually modeled by a Laplace probability density function, with parameter @l. A new method for @l estimation from quantized coefficient data is presented; it combines maximum-likelihood with linear prediction estimates, exploring the correlation between @l values at adjacent DCT frequencies. The resulting coefficient distributions are then used for estimating the local error due to lossy encoding. Local error estimates are also perceptually weighted, using a well-known perceptual model by Watson. When confronted with subjective quality evaluation data, results show that the quality scores that result from the proposed algorithm are well correlated with the human perception of quality. Since no knowledge about the original (reference) images is required, the proposed method resembles a no-reference quality metric for image evaluation. The quality of service limitation of today's Internet is a major challenge for real-time voice communications. Excessive delay, packet loss, and high delay jitter all impair the communication quality. A new receiver-based playout scheduling scheme is proposed to improve the tradeoff between buffering delay and late loss for real-time voice communication over IP networks. In this scheme the network delay is estimated from past statistics and the playout time of the voice packets is adaptively adjusted. In contrast to previous work, the adjustment is not only performed between talkspurts, but also within talkspurts in a highly dynamic way. Proper reconstruction of continuous playout speech is achieved by scaling individual voice packets using a time-scale modification technique based on the Waveform Similarity Overlap-Add (WSOLA) algorithm. Results of subjective listening tests show that this operation does not impair audio quality, since the adaptation process requires infrequent scaling of the voice packets and low playout jitter is perceptually tolerable. The same time-scale modification technique is also used to conceal packet loss at very low delay, i.e., one packet time. Simulation results based on Internet measurements show that the tradeoff between buffering delay and late loss can be improved significantly. The overall audio quality is investigated based on subjective listening tests, showing typical gains of 1 on a 5-point scale of the Mean Opinion Score.