This paper investigates a new learning formulation called structured sparsity, which is a natural extension of the standard sparsity concept in statistical learning and compressive sensing. By allowing arbitrary structures on the feature set, this concept generalizes the group sparsity idea that has become popular in recent years. A general theory is developed for learning with structured sparsity, based on the notion of coding complexity associated with the structure. It is shown that if the coding complexity of the target signal is small, then one can achieve improved performance by using coding complexity regularization methods, which generalize the standard sparse regularization. Moreover, a structured greedy algorithm is proposed to efficiently solve the structured sparsity problem. It is shown that the greedy algorithm approximately solves the coding complexity optimization problem under appropriate conditions. Experiments are included to demonstrate the advantage of structured sparsity over standard sparsity on some real applications.
Compressive sensing (CS) is an alternative to Shannon Nyquist sampling for the acquisition of sparse or compressible signals that can be well approximated by just K ? N elements from an N -dimensional basis. Instead of taking periodic samples, CS measures inner products with M < N random vectors and then recovers the signal via a sparsity-seeking optimization or greedy algorithm. Standard CS dictates that robust signal recovery is possible from M = O(K log(N K)) measurements. It is possible to substantially decrease M without sacrificing robustness by leveraging more realistic signal models that go beyond simple sparsity and compressibility by including structural dependencies between the values and locations of the signal coefficients. This paper introduces a model-based CS theory that parallels the conventional theory and provides concrete guidelines on how to create model-based recovery algorithms with provable performance guarantees. A highlight is the introduction of a new class of structured compressible signals along with a new sufficient condition for robust structured compressible signal recovery that we dub the restricted amplification property, which is the natural counterpart to the restricted isometry property of conventional CS. Two examples integrate two relevant signal models-wavelet trees and block sparsity-into two state-of-the-art CS recovery algorithms and prove that they offer robust recovery from just M = O(K) measurements. Extensive numerical simulations demonstrate the validity and applicability of our new theory and algorithms. A method and system for recording and reproducing, such as on a record carrier, content information and supplemental information relating thereto. The content information may be audio and or video, and the supplemental information may provide author identification and or copy control status. An encoded signal is generated representing the content information and which includes a watermark pattern representing the supplemental information. The watermark pattern cannot be changed without impairing the quality of the content information during reproduction. The supplemental information also includes a control pattern, the watermark being generated by applying a one-way function to such control pattern. This has the advantage that any alteration of the watermark or the control pattern can be detected easily, because it is not computationally feasible to calculate a new control pattern for an altered watermark. Therefore, the supplemental information is well protected against unauthorized manipulation. An attempt to fully replace the watermark pattern will affect the quality of reproduction of the content information. In a copy control method allowing a first generation copy ("copy-once"), the original control pattern is processed several times by the one-way function for generating the watermark. Each player or recorder processes the control pattern once before outputting recording it, thus forming a cryptographically protected down-counter. 