We propose a novel method for the blind separation of audio signals produced by musical instruments. While the approach of applying non-negative matrix factorization (NMF) has been studied in many papers, it does not make use of the pitch-invariance that the sounds of instruments exhibit. This limitation can be overcome by using tensor factorization, in which context the use of log-frequency spectrograms was initiated, but this still requires the specific tuning of the instruments to be hard-coded into the algorithm. We develop a time-frequency representation that is both shift-invariant and frequency-aligned, with a variant that can also be used for wideband signals. Our separation algorithm exploits this shift-invariance in order to find patterns of peaks related to specific instruments, while non-linear optimization enables it to represent arbitrary frequencies and incorporate inharmonicity, and the reasonability of the representation is ensured by a sparsity condition. The relative amplitudes of the harmonics are saved in a dictionary, which is trained via a modified version of ADAM. For a realistic monaural piece with acoustic recorder and violin, we achieve qualitatively good separation with a signal-to-distortion ratio (SDR) of 12.5 dB, a signal-to-interference ratio (SIR) of 25.7 dB, and a signal-to-artifacts ratio (SAR) of 12.7 dB, averaged.
The Heisenberg uncertainty principle (cf. @cite_8 @cite_6 ) sets a fundamental bound to the time-frequency resolution achievable. If we use Gaussian windows where @math is the standard deviation of the window in the time domain and @math is the standard deviation of its Fourier transform, then we have @math . If, for instance, @math , then @math . For speech processing, this may be sufficient, but considering music that contains frequencies as low as @math , this deviation equals almost half an octave (or a tritone).
@cite_6 @cite_8
2015064056 1605417594
For a positive number @math the Mallows measure on the symmetric group is the probability measure on @math such that @math is proportional to @math -to-the-power- @math where @math equals the number of inversions: @math equals the number of pairs @math . One may consider this as a mean-field model from statistical mechanics. The weak large deviation principle may replace the Gibbs variational principle for characterizing equilibrium measures. In this sense, we prove absence of phase transition, i.e., phase uniqueness. We introduce a new low-distortion embedding of @math into @math ( @math ) called the fast Johnson-Lindenstrauss transform (FJLT). The FJLT is faster than standard random projections and just as easy to implement. It is based upon the preconditioning of a sparse projection matrix with a randomized Fourier transform. Sparse random projections are unsuitable for low-distortion embeddings. We overcome this handicap by exploiting the “Heisenberg principle” of the Fourier transform, i.e., its local-global duality. The FJLT can be used to speed up search algorithms based on low-distortion embeddings in @math and @math . We consider the case of approximate nearest neighbors in @math . We provide a faster algorithm using classical projections, which we then speed up further by plugging in the FJLT. We also give a faster algorithm for searching over the hypercube. The k-means method is a widely used clustering algorithm. One of its distinguished features is its speed in practice. Its worst-case running-time, however, is exponential, leaving a gap between practical and theoretical performance. Arthur and Vassilvitskii (FOCS 2006) aimed at closing this gap, and they proved a bound of @math on the smoothed running-time of the k-means method, where n is the number of data points and @math is the standard deviation of the Gaussian perturbation. This bound, though better than the worst-case bound, is still much larger than the running-time observed in practice. We improve the smoothed analysis of the k-means method by showing two upper bounds on the expected running-time of k-means. First, we prove that the expected running-time is bounded by a polynomial in @math and @math . Second, we prove an upper bound of @math , where d is the dimension of the data space. The polynomial is independent of k and d, and we obtain a polynomial bound for the expected running-time for @math . Finally, we show that k-means runs in smoothed polynomial time for one-dimensional instances. Super-resolution refers to the process of recovering the locations and amplitudes of a collection of point sources, represented as a discrete measure, given @math of its noisy low-frequency Fourier coefficients. The recovery process is highly sensitive to noise whenever the distance @math between the two closest point sources is less than @math . This paper studies the fundamental difficulty of super-resolution and the performance guarantees of a subspace method called MUSIC in the regime that @math . The most important quantity in our theory is the minimum singular value of the Vandermonde matrix whose nodes are specified by the source locations. Under the assumption that the nodes are closely spaced within several well-separated clumps, we derive a sharp and non-asymptotic lower bound for this quantity. Our estimate is given as a weighted @math sum, where each term only depends on the configuration of each individual clump. This implies that, as the noise increases, the super-resolution capability of MUSIC degrades according to a power law where the exponent depends on the cardinality of the largest clump. Numerical experiments validate our theoretical bounds for the minimum singular value and the resolution limit of MUSIC. When there are @math point sources located on a grid with spacing @math , the fundamental difficulty of super-resolution can be quantitatively characterized by a min-max error, which is the reconstruction error incurred by the best possible algorithm in the worst-case scenario. We show that the min-max error is closely related to the minimum singular value of Vandermonde matrices, and we provide a non-asymptotic and sharp estimate for the min-max error, where the dominant term is @math . This paper demonstrates that convolution with random waveform followed by random time-domain subsampling is a universally efficient compressive sensing strategy. We show that an @math -dimensional signal which is @math -sparse in any fixed orthonormal representation can be recovered from @math samples from its convolution with a pulse whose Fourier transform has unit magnitude and random phase at all frequencies. The time-domain subsampling can be done in one of two ways: in the first, we simply observe @math samples of the random convolution; in the second, we break the random convolution into @math blocks and summarize each with a single randomized sum. We also discuss several imaging applications where convolution with a random pulse allows us to superresolve fine-scale features, allowing us to recover high-resolution signals from low-resolution measurements. We consider @math Hermitian matrices with i.i.d. entries @math whose tail probabilities @math behave like @math for some @math and @math . We establish a large deviation principle for the empirical spectral measure of @math with speed @math with a good rate function @math that is finite only if @math is of the form @math for some probability measure @math on @math , where @math denotes the free convolution and @math is Wigner's semicircle law. We obtain explicit expressions for @math in terms of the @math th moment of @math . The proof is based on the analysis of large deviations for the empirical distribution of very sparse random rooted networks. Consider the problem of recovering a measure @math supported on a lattice of span @math , when measurements are only available concerning the Fourier Transform @math at frequencies @math . If @math is much smaller than the Nyquist frequency @math and the measurements are noisy, then, in general, stable recovery of @math is impossible. In this paper it is shown that if, in addition, we know that the measure @math satisfies certain sparsity constraints, then stable recovery is possible. Say that a set has Rayleigh index less than or equal to R if in any interval of length @math there are at most R elements. Indeed, if the (unknown) support of @math is known, a priori, to have Rayleigh index at most R, then stable recovery is possible with a stability coefficient that grows at most like @math as @math . This result validates certain practical efforts, in spectroscopy, seismic prospecting, and astrono... It is shown that statistics of records for time series generated by random walks are independent of the details of the jump distribution, as long as the latter is continuous and symmetric. In @math steps, the mean of the record distribution grows as the @math while the standard deviation grows as @math , so the distribution is non-self-averaging. The mean shortest and longest duration records grow as @math and @math , respectively. The case of a discrete random walker is also studied, and similar asymptotic behavior is found. Let @math be a compact Riemannian submanifold of @math of dimension @math and let @math be a sample of i.i.d. points in @math with uniform distribution. We study the random operators @math where @math is the Gaussian kernel and @math as @math Such operators can be viewed as graph laplacians (for a weighted graph with vertices at data points) and they have been used in the machine learning literature to approximate the Laplace-Beltrami operator of @math @math (divided by the Riemannian volume of the manifold). We prove several results on a.s. and distributional convergence of the deviations @math for smooth functions @math both pointwise and uniformly in @math and @math (here @math and @math is the Riemannian volume measure). In particular, we show that for any class @math of three times differentiable functions on @math with uniformly bounded derivatives @math as soon as @math and also prove asymptotic normality of @math (functional CLT) for a fixed @math and uniformly in @math There has been much progress on efficient algorithms for clustering data points generated by a mixture of @math probability distributions under the assumption that the means of the distributions are well-separated, i.e., the distance between the means of any two distributions is at least @math standard deviations. These results generally make heavy use of the generative model and particular properties of the distributions. In this paper, we show that a simple clustering algorithm works without assuming any generative (probabilistic) model. Our only assumption is what we call a "proximity condition": the projection of any data point onto the line joining its cluster center to any other cluster center is @math standard deviations closer to its own center than the other center. Here the notion of standard deviations is based on the spectral norm of the matrix whose rows represent the difference between a point and the mean of the cluster to which it belongs. We show that in the generative models studied, our proximity condition is satisfied and so we are able to derive most known results for generative models as corollaries of our main result. We also prove some new results for generative models - e.g., we can cluster all but a small fraction of points only assuming a bound on the variance. Our algorithm relies on the well known @math -means algorithm, and along the way, we prove a result of independent interest -- that the @math -means algorithm converges to the "true centers" even in the presence of spurious points provided the initial (estimated) centers are close enough to the corresponding actual centers and all but a small fraction of the points satisfy the proximity condition. Finally, we present a new technique for boosting the ratio of inter-center separation to standard deviation. Using large deviation results for a uniformly random permutation in @math , asymptotics of normalizing constants are computed and limits of permutations obtained for some non uniform distributions on @math . A pseudo-likelihood type estimator is shown to be consistent in a class of one parameter exponential families on permutations. A new proof of the large deviation principle of a uniformly random permutation on @math is given. An algorithm of the form @math , where @math is a smooth function on @math , @math is a sequence of @math -valued random variables, @math is a sequence of independent standard d-dimensional Gaussian random variables, @math and @math for k large, is considered. An algorithm of this type arises by adding slowly decreasing white Gaussian noise to a stochastic gradient algorithm. It is shown, under suitable conditions on @math , @math , A, and B, that @math converges in probability to the set of global minima of @math . No prior information is assumed as to what bounded region contains a global minimum. The analysis is based on the asymptotic behavior of the related diffusion process @math , where @math is a standard d-dimensional Wiener process and @math for t large. In this paper, we explore theoretical properties of training a two-layered ReLU network @math with centered @math -dimensional spherical Gaussian input @math ( @math =ReLU). We train our network with gradient descent on @math to mimic the output of a teacher network with the same architecture and fixed parameters @math . We show that its population gradient has an analytical formula, leading to interesting theoretical analysis of critical points and convergence behaviors. First, we prove that critical points outside the hyperplane spanned by the teacher parameters ("out-of-plane") are not isolated and form manifolds, and characterize in-plane critical-point-free regions for two ReLU case. On the other hand, convergence to @math for one ReLU node is guaranteed with at least @math probability, if weights are initialized randomly with standard deviation upper-bounded by @math , consistent with empirical practice. For network with many ReLU nodes, we prove that an infinitesimal perturbation of weight initialization results in convergence towards @math (or its permutation), a phenomenon known as spontaneous symmetric-breaking (SSB) in physics. We assume no independence of ReLU activations. Simulation verifies our findings. The problem of finding large average submatrices of a real-valued matrix arises in the exploratory analysis of data from a variety of disciplines, ranging from genomics to social sciences. In this paper we provide a detailed asymptotic analysis of large average submatrices of an @math Gaussian random matrix. The first part of the paper addresses global maxima. For fixed @math we identify the average and the joint distribution of the @math submatrix having largest average value. As a dual result, we establish that the size of the largest square sub-matrix with average bigger than a fixed positive constant is, with high probability, equal to one of two consecutive integers that depend on the threshold and the matrix dimension @math . The second part of the paper addresses local maxima. Specifically we consider submatrices with dominant row and column sums that arise as the local optima of iterative search procedures for large average submatrices. For fixed @math , we identify the limiting average value and joint distribution of a @math submatrix conditioned to be a local maxima. In order to understand the density of such local optima and explain the quick convergence of such iterative procedures, we analyze the number @math of local maxima, beginning with exact asymptotic expressions for the mean and fluctuation behavior of @math . For fixed @math , the mean of @math is @math while the standard deviation is @math . Our principal result is a Gaussian central limit theorem for @math that is based on a new variant of Stein's method. While spike trains are obviously not band-limited, the theory of super-resolution tells us that perfect recovery of unknown spike locations and weights from low-pass Fourier transform measurements is possible provided that the minimum spacing, ( ), between spikes is not too small. Specifically, for a measurement cutoff frequency of (f_c ), Donoho (SIAM J Math Anal 23(5):1303–1331, 1992) showed that exact recovery is possible if the spikes (on ( R )) lie on a lattice and ( > 1 f_c ), but does not specify a corresponding recovery method. Candes and Fernandez-Granda (Commun Pure Appl Math 67(6):906–956, 2014; Inform Inference 5(3):251–303, 2016) provide a convex programming method for the recovery of periodic spike trains (i.e., spike trains on the torus ( T )), which succeeds provably if ( > 2 f_c ) and (f_c 128 ) or if ( > 1.26 f_c ) and (f_c 10^3 ), and does not need the spikes within the fundamental period to lie on a lattice. In this paper, we develop a theory of super-resolution from short-time Fourier transform (STFT) measurements. Specifically, we present a recovery method similar in spirit to the one in Candes and Fernandez-Granda (2014) for pure Fourier measurements. For a STFT Gaussian window function of width ( = 1 (4f_c) ) this method succeeds provably if ( > 1 f_c ), without restrictions on (f_c ). Our theory is based on a measure-theoretic formulation of the recovery problem, which leads to considerable generality in the sense of the results being grid-free and applying to spike trains on both ( R ) and ( T ). The case of spike trains on ( R ) comes with significant technical challenges. For recovery of spike trains on ( T ) we prove that the correct solution can be approximated—in weak-* topology—by solving a sequence of finite-dimensional convex programming problems. We introduce a general purpose algorithm for rapidly computing certain types of oscillatory integrals which frequently arise in problems connected to wave propagation, general hyperbolic equations, and curvilinear tomography. The problem is to numerically evaluate a so-called Fourier integral operator (FIO) of the form @math at points given on a Cartesian grid. Here, @math is a frequency variable, @math is the Fourier transform of the input @math , @math is an amplitude, and @math is a phase function, which is typically as large as @math ; hence the integral is highly oscillatory. Because a FIO is a dense matrix, a naive matrix vector product with an input given on a Cartesian grid of size @math by @math would require @math operations. This paper develops a new numerical algorithm which requires @math operations and as low as @math in storage space (the constants in front of these estimates are small). It operates by localizing the integral over polar wedges with small angular aperture in the frequency plane. On each wedge, the algorithm factorizes the kernel @math into two components: (1) a diffeomorphism which is handled by means of a nonuniform FFT and (2) a residual factor which is handled by numerical separation of the spatial and frequency variables. The key to the complexity and accuracy estimates is the fact that the separation rank of the residual kernel is provably independent of the problem size. Several numerical examples demonstrate the numerical accuracy and low computational complexity of the proposed methodology. We also discuss the potential of our ideas for various applications such as reflection seismology. We introduce a complete parameterization of the family of two-dimensional steerable wavelets that are polar-separable in the Fourier domain under the constraint of self-reversibility. These wavelets are constructed by multiorder generalized Riesz transformation of a primary isotropic bandpass pyramid. The backbone of the transform (pyramid) is characterized by a radial frequency profile function @math , while the directional wavelet components at each scale are encoded by an @math shaping matrix @math , where @math is the number of wavelet channels and @math the order of the Riesz transform. We provide general conditions on @math and @math for the underlying wavelet system to form a tight frame of @math (with a redundancy factor @math ). The proposed framework ensures that the wavelets are steerable and provides new degrees of freedom (shaping matrix @math ) that can be exploited for designing specific wavelet systems. It encompasses many known transforms as part... We prove that low-rank matrices can be recovered efficiently from a small number of measurements that are sampled from orbits of a certain matrix group. As a special case, our theory makes statements about the phase retrieval problem. Here, the task is to recover a vector given only the amplitudes of its inner product with a small number of vectors from an orbit. Variants of the group in question have appeared under different names in many areas of mathematics. In coding theory and quantum information, it is the complex Clifford group; in time-frequency analysis the oscillator group; and in mathematical physics the metaplectic group. It affords one particularly small and highly structured orbit that includes and generalizes the discrete Fourier basis: While the Fourier vectors have coefficients of constant modulus and phases that depend linearly on their index, the vectors in said orbit have phases with a quadratic dependence. In quantum information, the orbit is used extensively and is known as the set of stabilizer states. We argue that due to their rich geometric structure and their near-optimal recovery properties, stabilizer states form an ideal model for structured measurements for phase retrieval. Our results hold for @math measurements, where the oversampling factor k varies between @math and @math depending on the orbit. The reconstruction is stable towards both additive noise and deviations from the assumption of low rank. If the matrices of interest are in addition positive semidefinite, reconstruction may be performed by a simple constrained least squares regression. Our proof methods could be adapted to cover orbits of other groups. We investigate the minimization of Newton's functional for the problem of the body of minimal resistance with maximal height @math cite butt in the class of convex developable functions defined in a disc. This class is a natural candidate to find a (non-radial) minimizer in accordance with the results of cite lrp2. We prove that the minimizer in this class has a minimal set in the form of a regular polygon with @math sides centered in the disc, and numerical experiments indicate that the natural number @math is a non-decreasing function of @math . The corresponding functions all achieve a lower value of the functional than the optimal radially symmetric function with the same height @math . The convergence of a class of Metropolis-type Markov-chain annealing algorithms for global optimization of a smooth function @math on @math is established. No prior information is assumed as to what bounded region contains a global minimum. The analysis contained herein is based on writing the Metropolis-type algorithm in the form of a recursive stochastic algorithm @math , where @math is a standard white Gaussian sequence, @math are random variables, and @math , @math for k large. Convergence results for @math are then applied from our previous work [SIAM Journal on Control and Optimization, 29 (1991), pp. 999–1018]. Since the analysis of @math is based on the asymptotic behavior of the related Langevin-type Markov diffusion annealing algorithm @math , where @math is a standard Wiener process and $c(t) = ...