We propose an audio codec that addresses the low-delay requirements of some applications such as network music performance. The codec is based on the modified discrete cosine transform (MDCT) with very short frames and uses gain-shape quantization to preserve the spectral envelope. The short frame sizes required for low delay typically hinder the performance of transform codecs. However, at 96 kbit s and with only 4 ms algorithmic delay, the proposed codec out-performs the ULD codec operating at the same rate. The total complexity of the codec is small, at only 17 WMOPS for real-time operation at 48 kHz.
The Fraunhofer Ultra Low Delay (ULD) codec @cite_10 is one of the only full-bandwidth codecs with an algorithmic delay comparable to the proposed codec. Its structure, however, is completely different from that of the proposed codec. ULD is based on time-domain linear prediction instead of the MDCT. It uses a pre-filter post-filter combination, whose parameters are transmitted in the bitstream, to shape the quantization noise. ULD frames are 128 samples with 128 samples of look-ahead, for a total algorithmic delay of 256 samples at 48 kHz (5.3 ms). One disadvantage of the linear-prediction approach is the difficulty of resynchronizing the decoder after a packet is lost @cite_9 . In contrast, the proposed codec only uses inter-frame prediction for @math , so the decoder resynchronizes very quickly after packet loss. Changing the proposed codec to have completely independent packets would cost approximately 12 bits per frame.
@cite_9 @cite_10
2120871939 2166509549 204827909 2111085154
This paper proposes a versatile perceptual audio coding method that achieves high compression ratios and is capable of low encoding decoding delay. It accommodates a variety of source signals (including both music and speech) with different sampling rates. It is based on separating irrelevance and redundancy reductions into independent functional units. This contrasts traditional audio coding where both are integrated within the same subband decomposition. The separation allows for the independent optimization of the irrelevance and redundancy reduction units. For both reductions, we rely on adaptive filtering and predictive coding as much as possible to minimize the delay. A psycho-acoustically controlled adaptive linear filter is used for the irrelevance reduction, and the redundancy reduction is carried out by a predictive lossless coding scheme, which is termed weighted cascaded least mean squared (WCLMS) method. Experiments are carried out on a database of moderate size which contains mono-signals of different sampling rates and varying nature (music, speech, or mixed). They show that the proposed WCLMS lossless coder outperforms other competing lossless coders in terms of compression ratios and delay, as applied to the pre-filtered signal. Moreover, a subjective listening test of the combined pre-filter lossless coder and a state-of-the-art perceptual audio coder (PAC) shows that the new method achieves a comparable compression ratio and audio quality with a lower delay. In this paper we present and evaluate several concealment strategies for packet losses in the context of a low delay predictive audio coder. Our goal is to minimize the audible impact of a packet loss. The problem is that the predictive coder is backward adaptive, hence depending on past values. There is a predictor reset, but to increase coding efficiency, the distance between two resets is several hundred packets. Hence, not only the lost packet itself cannot be reconstructed, but the transmitted data up to the next reset does not result in an exact reconstruction of the audio signal. Our approach is to try to use as much information as possible from the still available data until the next reset, and to reconstruct an audio signal such that distortions are least objectionable. To compare different approaches, we conducted a listening test. The result is that an adaptive reconstruction filter works best in this context. Real-world video sequences coded at low bit rates suffer from compression artifacts, which are visually disruptive and can cause problems to computer vision algorithms. Unlike the denoising problem where the high frequency components of the signal are present in the noisy observation, most high frequency details are lost during compression and artificial discontinuities arise across the coding block boundaries. In addition to sparse spatial priors that can reduce the blocking artifacts for a single frame, temporal information is needed to recover the lost spatial details. However, establishing accurate temporal correspondences from the compressed videos is challenging because of the loss of high frequency details and the increase of false blocking artifacts. In this paper, we propose a non-causal temporal prior model to reduce video compression artifacts by propagating information from adjacent frames and iterating between image reconstruction and motion estimation. Experimental results on real-world sequences demonstrate that the deblocked videos by the proposed system have marginal statistics of high frequency components closer to those of the original ones, and are better input for standard edge and corner detectors than the coded ones. In existing pyramid-based spatially scalable coding schemes, such as H.264 MPEG-4 SVC (scalable video coding), video frame at a certain high-resolution layer is mainly predicted either from the same frame at the next lower resolution layer, or from the temporal neighboring frames within the same resolution layer. But these schemes fail to exploit both kinds of correlation simultaneously and therefore cannot remove the redundancies among resolution layers efficiently. This paper extends the idea of spatiotemporal subband transform and proposes a general in-scale motion compensation technique for pyramid-based spatially scalable video coding. Video frame at each high-resolution layer is partitioned into two parts in frequency. Prediction for the lowpass part is derived from the next lower resolution layer, whereas prediction for the highpass part is obtained from neighboring frames within the same resolution layer, to further utilize temporal correlation. In this way, both kinds of correlation are exploited simultaneously and the cross-resolution-layer redundancy can be highly removed. Furthermore, this paper also proposes a macroblock-based adaptive in-scale technique for hybrid spatial and SNR scalability. Experimental results show that the proposed techniques can significantly improve the spatial scalability performance of H.264 MPEG-4 SVC, especially when the bit-rate ratio of lower resolution bit stream to higher resolution bit stream is considerable.