Given samples from a group of related regression tasks, a data-enriched model describes observations by a common and per-group individual parameters. In high-dimensional regime, each parameter has its own structure such as sparsity or group sparsity. In this paper, we consider the general form of data enrichment where data comes in a fixed but arbitrary number of tasks $G$ and any convex function, e.g., norm, can characterize the structure of both common and individual parameters. 	 We propose an estimator for the high-dimensional data enriched model and investigate its statistical properties.   We delineate the sample complexity of our estimator and provide high probability non-asymptotic bound for estimation error of all parameters under a condition weaker than the state-of-the-art. We propose an iterative estimation algorithm with a geometric convergence rate. Overall, we present a first through statistical and computational analysis of inference in the data enriched model. 
	 Over the past two decades, major advances have been made in estimating structured parameters, e.g., sparse, low-rank, etc., in high-dimensional small sample problems BID13 BID6 BID14 . Such estimators consider a suitable (semi) parametric model of the response: y = φ(x, β * )+ω based on n samples {(x i , y i )} n i=1and β * ∈ R p is the true parameter of interest. The unique aspect of such high-dimensional setup is that the number of samples n < p, and the structure in β * , e.g., sparsity, low-rank, makes the estimation possible (Tibshirani, 1996; BID7 BID5 ). In several real world problems, natural grouping among samples arises and learning a single common model β 0 for all samples or many per group individual models β g s are unrealistic. The middle ground model for such a scenario is the superposition of common and individual parameters β 0 + β g which has been of recent interest in the statistical machine learning community BID16 and is known by multiple names. It is a form of multi-task learning (Zhang & Yang, 2017; BID17 when we consider regression in each group as a task. It is also called data sharing BID15 since information contained in different group is shared through the common parameter β 0 . And finally, it has been called data enrichment BID10 BID0 because we enrich our data set with pooling multiple samples from different but related sources.In this paper, we consider the following data enrichment (DE) model where there is a common parameter β * 0 shared between all groups plus individual per-group parameters β * g which characterize the deviation of group g: y gi = φ(x gi , (β * 0 + β * g )) + ω gi , g ∈ {1, . . . , G}, (1) where g and i index the group and samples respectively. Note that the DE model is a system of coupled superposition models. We specifically focus on the high-dimensional small sample regime for (1) where the number of samples n g for each group is much smaller than the ambient dimensionality, i.e., ∀g : n g p. Similar to all other highdimensional models, we assume that the parameters β g are structured, i.e., for suitable convex functions f g 's, f g (β g ) is small. Further, for the technical analysis and proofs, we focus on the case of linear models, i.e., φ(x, β) = x T β. The results seamlessly extend to more general non-linear models, e.g., generalized linear models, broad families of semi-parametric and single-index models, non-convex models, etc., using existing results, i.e., how models like LASSO have been extended (e.g. employing ideas such as restricted strong convexity (Negahban & Wainwright, 2012) ).In the context of Multi-task learning (MTL), similar models have been proposed which has the general form of y gi = x T gi (β * 1g + β * 2g ) + ω gi where B 1 = [β 11 , . . . , β 1G ] and B 2 = [β 21 , . . . , β 2G ] are two parameter matrices (Zhang & Yang, 2017) . To capture relation of tasks, different types of constraints are assumed for parameter matrices. For example, BID11 assumes B 1 and B 2 are sparse and low rank respectively. In this parameter matrix decomposition framework for MLT, the most related work to ours is the one proposed by BID17 where authors regularize the regression with B 1 1,∞ and B 2 1,1 where norms are p, q-norms on rows of matrices. Parameters of B 1 are more general than DE's common parameter when we use f 0 (β 0 ) = β 0 1 . This is because B 1 1,∞ regularizer enforces shared support of β * 1g s, i.e., supp(β * 1i ) = supp(β * 1j ) but allows β * 1i = β * 1j . Further sparse variation between parameters of different tasks is induced by B 2 1,1 which has an equivalent effect to DE's individual parameters where f g (·)s are l 1 -norm. Our analysis of DE framework suggests that it is more data efficient than this setup of BID17 ) because they require every task i to have large enough samples to learn its own common parameters β i while DE shares the common parameter and only requires the total dataset over all tasks to be sufficiently large.The DE model where β g 's are sparse has recently gained attention because of its application in wide range of domains such as personalized medicine BID12 , sentiment analysis, banking strategy BID15 , single cell data analysis (Ollier & Viallon, 2015) , road safety (Ollier & Viallon, 2014) , and disease subtype analysis BID12 . In spite of the recent surge in applying data enrichment framework to different domains, limited advances have been made in understanding the statistical and computational properties of suitable estimators for the data enriched model. In fact, non-asymptotic statistical properties, including sample complexity and statistical rates of convergence, of regularized estimators for the data enriched model is still an open question BID15 Ollier & Viallon, 2014) . To the best of our knowledge , the only theoretical guarantee for data enrichment is provided in (Ollier & Viallon, 2015) where authors prove sparsistency of their proposed method under the stringent irrepresentability condition of the design matrix for recovering supports of common and individual parameters. Existing support recovery guarantees (Ollier & Viallon, 2015) , sample complexity and l 2 consistency results BID17 of related models are restricted to sparsity and l 1 -norm, while our estimator and norm consistency analysis work for any structure induced by arbitrary convex functions f g . Moreover, no computational results, such as rates of convergence of the optimization algorithms associated with proposed estimators, exist in the literature.
We provide an estimator and an estimation algorithm for a class of multi-task regression problem and provide statistical and computational analysis..
{"src": "scitldr-AIC_train", "paper_id": "Byx1S9Bj3N"}