We present a novel family of language model (LM) estimation techniques named Sparse Non-negative Matrix (SNM) estimation. A first set of experiments empirically evaluating it on the On e Billion Word Benchmark [, 2013] shows that SNM n-gram LMs perform almost as well as the well-established Kneser-Ney (KN) models. When using skip-gram features the models are able to match the state-of-the-art recurrent neural network (RNN) LMs; combining the two modeling techniques yields the best known result on the benchmark. The computational advantages of SNM over both maximum entropy and RNN LM estimation are probably its main strength, promising an approach that has the same flexibility in combining arbitrary feature s effectively and yet should scale to very large amounts of data as gracefully as n-gram LMs do.
SNM estimation is closely related to all @math -gram LM smoothing techniques that rely on mixing relative frequencies at various orders. Unlike most of those, it combines the predictors at various orders without relying on a hierarchical nesting of the contexts, setting it closer to the family of maximum entropy (ME) @cite_13 , or exponential models.
@cite_13
2021196544 2032665768 2950162031 2064658852
This paper addresses the problem of sound source separation from a multichannel microphone array capture via estimation of source spatial covariance matrix (SCM) of a short-time Fourier transformed mixture signal. In many conventional audio separation algorithms the source mixing parameter estimation is done separately for each frequency thus making them prone to errors and leading to suboptimal source estimates. In this paper we propose a SCM model which consists of a weighted sum of direction of arrival (DoA) kernels and estimate only the weights dependent on the source directions. In the proposed algorithm, the spatial properties of the sources become jointly optimized over all frequencies, leading to more coherent source estimates and mitigating the effect of spatial aliasing at high frequencies. The proposed SCM model is combined with a linear model for magnitudes and the parameter estimation is formulated in a complex-valued non-negative matrix factorization (CNMF) framework. Simulations consist of recordings done with a hand-held device sized array having multiple microphones embedded inside the device casing. Separation quality of the proposed algorithm is shown to exceed the performance of existing state of the art separation methods with two sources when evaluated by objective separation quality metrics. In this paper, a novel algorithm to blindly separate an instantaneous linear underdetermined mixture of nonstationary sources is proposed. It means that the number of sources exceeds the number of channels of the available data. The separation is based on the working assumption that the sources are piecewise stationary with a different variance in each block. It proceeds in two steps: 1) estimating the mixing matrix, and 2) computing the optimum beamformer in each block to maximize the signal-to-interference ratio of each separated signal with respect to the remaining signals. Estimating the mixing matrix is accomplished through a specialized tensor decomposition of the set of sample covariance matrices of the received mixture in each block. It utilizes optimum weighting, which allows statistically efficient (CRB attaining) estimation provided that the data obey the assumed Gaussian piecewise stationary model. In simulations, performance of the algorithm is successfully tested on blind separation of 16 speech signals from nine linear instantaneous mixtures of these signals. We investigate the clustering performances of the relaxed @math means in the setting of sub-Gaussian Mixture Model (sGMM) and Stochastic Block Model (SBM). After identifying the appropriate signal-to-noise ratio (SNR), we prove that the misclassification error decay exponentially fast with respect to this SNR. These partial recovery bounds for the relaxed @math means improve upon results currently known in the sGMM setting. In the SBM setting, applying the relaxed @math means SDP allows to handle general connection probabilities whereas other SDPs investigated in the literature are restricted to the assortative case (where within group probabilities are larger than between group probabilities). Again, this partial recovery bound complements the state-of-the-art results. All together, these results put forward the versatility of the relaxed @math means. Abstract In this paper, a hard thresholding wavelet estimator is constructed for a deconvolution model in a periodic setting that has long-range dependent noise. The estimation paradigm is based on a maxiset method that attains a near optimal rate of convergence for a variety of L p loss functions and a wide variety of Besov spaces in the presence of strong dependence. The effect of long-range dependence is detrimental to the rate of convergence. The method is implemented using a modification of the WaveD -package in R and an extensive numerical study is conducted. The numerical study supplements the theoretical results and compares the LRD estimator with a naive application of the standard WaveD approach.