This paper considers recovering @math -dimensional vectors @math , and @math from their circular convolutions @math . The vector @math is assumed to be @math -sparse in a known basis that is spread out in the Fourier domain, and each input @math is a member of a known @math -dimensional random subspace. We prove that whenever @math , the problem can be solved effectively by using only the nuclear-norm minimization as the convex relaxation, as long as the inputs are sufficiently diverse and obey @math . By "diverse inputs", we mean that the @math 's belong to different, generic subspaces. To our knowledge, this is the first theoretical result on blind deconvolution where the subspace to which @math belongs is not fixed, but needs to be determined. We discuss the result in the context of multipath channel estimation in wireless communications. Both the fading coefficients, and the delays in the channel impulse response @math are unknown. The encoder codes the @math -dimensional message vectors randomly and then transmits coded messages @math 's over a fixed channel one after the other. The decoder then discovers all of the messages and the channel response when the number of samples taken for each received message are roughly greater than @math , and the number of messages is roughly at least @math .
An important recent article from the same group settles the recovery guarantee for a regularized gradient descent algorithm for blind deconvolution, in the single-input case and with the scaling @math @cite_29 . This result, however, makes the assumption of a fixed subspace for the sparse impulse response. Note that gradient descent algorithms are expected to have much more favorable runtimes than semidefinite programming, when their basin of attraction can be established to be wide enough, as in @cite_29 .
@cite_29
2430435693 2248294424 2896834587 2607406448
We study the question of reconstructing two signals @math and @math from their convolution @math . This problem, known as blind deconvolution , pervades many areas of science and technology, including astronomy, medical imaging, optics, and wireless communications. A key challenge of this intricate non-convex optimization problem is that it might exhibit many local minima. We present an efficient numerical algorithm that is guaranteed to recover the exact solution, when the number of measurements is (up to log-factors) slightly larger than the information-theoretical minimum, and under reasonable conditions on @math and @math . The proposed regularized gradient descent algorithm converges at a geometric rate and is provably robust in the presence of noise. To the best of our knowledge, our algorithm is the first blind deconvolution algorithm that is numerically efficient, robust against noise, and comes with rigorous recovery guarantees under certain subspace conditions. Moreover, numerical experiments do not only provide empirical verification of our theory, but they also demonstrate that our method yields excellent performance even in situations beyond our theoretical framework. This note considers the problem of blind identification of a linear, time-invariant (LTI) system when the input signals are unknown, but belong to sufficiently diverse, known subspaces. This problem can be recast as the recovery of a rank-1 matrix, and is effectively relaxed using a semidefinite program (SDP). We show that exact recovery of both the unknown impulse response, and the unknown inputs, occurs when the following conditions are met: (1) the impulse response function is spread in the Fourier domain, and (2) the N input vectors belong to generic, known subspaces of dimension K in ℝL. Recent results in the well-understood area of low-rank recovery from underdetermined linear measurements can be adapted to show that exact recovery occurs with high probablility (on the genericity of the subspaces) provided that K,L, and N obey the information-theoretic scalings, namely L ≳ K and N ≳ 1 up to log factors. We study the implicit bias of gradient descent methods in solving a binary classification problem over a linearly separable dataset. The classifier is described by a nonlinear ReLU model and the objective function adopts the exponential loss function. We first characterize the landscape of the loss function and show that there can exist spurious asymptotic local minima besides asymptotic global minima. We then show that gradient descent (GD) can converge to either a global or a local max-margin direction, or may diverge from the desired max-margin direction in a general context. For stochastic gradient descent (SGD), we show that it converges in expectation to either the global or the local max-margin direction if SGD converges. We further explore the implicit bias of these algorithms in learning a multi-neuron network under certain stationary conditions, and show that the learned classifier maximizes the margins of each sample pattern partition under the ReLU activation. We propose a partially learned approach for the solution of ill-posed inverse problems with not necessarily linear forward operators. The method builds on ideas from classical regularisation theory and recent advances in deep learning to perform learning while making use of prior information about the inverse problem encoded in the forward operator, noise model and a regularising functional. The method results in a gradient-like iterative scheme, where the 'gradient' component is learned using a convolutional network that includes the gradients of the data discrepancy and regulariser as input in each iteration. We present results of such a partially learned gradient scheme on a non-linear tomographic inversion problem with simulated data from both the Sheep-Logan phantom as well as a head CT. The outcome is compared against filtered backprojection and total variation reconstruction and the proposed method provides a 5.4 dB PSNR improvement over the total variation reconstruction while being significantly faster, giving reconstructions of pixel images in about 0.4 s using a single graphics processing unit (GPU).