Semi-supervised Inference for Explained Variance in High-dimensional Linear Regression and Its Applications

We consider statistical inference for the explained variance $\beta^{\intercal}\Sigma \beta$ under the high-dimensional linear model $Y=X\beta+\epsilon$ in the semi-supervised setting, where $\beta$ is the regression vector and $\Sigma$ is the design covariance matrix. A calibrated estimator, which efficiently integrates both labelled and unlabelled data, is proposed. It is shown that the estimator achieves the minimax optimal rate of convergence in the general semi-supervised framework. The optimality result characterizes how the unlabelled data affects the minimax optimal rate. Moreover, the limiting distribution for the proposed estimator is established and data-driven confidence intervals for the explained variance are constructed. We further develop a randomized calibration technique for statistical inference in the presence of weak signals and apply the obtained inference results to a range of important statistical problems, including signal detection and global testing, prediction accuracy evaluation, and confidence ball construction. The numerical performance of the proposed methodology is demonstrated in simulation studies and an analysis of estimating heritability for a yeast segregant data set with multiple traits.


Introduction
High-dimensional linear models are ubiquitous in contemporary statistical modeling with a wide range of applications in many scientific fields. The early focus has been mainly on developing methods for the recovery of the whole regression vector via penalized or constrained ℓ 1 minimization approaches. Examples include the Lasso [Tibshirani, 1996], Dantzig Selector [Candès and Tao, 2007], MCP [Zhang, 2010], square-root Lasso [Belloni et al., 2011], and scaled Lasso [Sun and Zhang, 2012]. There have been significant recent interests in statistical inference for lowdimensional functionals, including confidence intervals and hypothesis testing for individual regression coefficients [Zhang and Zhang, 2014, van de Geer et al., 2014, Javanmard and Montanari, 2014a, minimaxity and adaptivity of confidence intervals for general linear functionals [Cai and Guo, 2017c], estimation of the signalto-noise-ratio [Verzelen andGassiat, 2016, Janson et al., 2015], inference for the ℓ q arXiv:1806.06179v1 [stat.ME] 16 Jun 2018 accuracy of a given estimator [Cai and Guo, 2017a], and estimation of quadratic functionals [Janson et al., 2015, Guo et al., 2017b.
Motivated by a range of applications, the present paper considers statistical inference for the explained variance, which is a one-dimensional weighted quadratic functional, in the high-dimensional and semi-supervised setting. We first develop in detail the theory for optimal estimation of the explained variance, which also leads to the construction of confidence intervals. The results are then applied to several other important statistical inference problems.

Problem Formulation and Motivations
We consider the high-dimensional linear model with a random design, where y i ∈ R and X i· ∈ R p denote respectively the outcome and the measured covariates of the i-th observation, ϵ i denotes the error and β ∈ R p denotes the highdimensional regression vector. The rows X i· are i.i.d. p-dimensional sub-gaussian random vectors with mean 0 and covariance matrix Σ and the errors {ϵ i } 1≤i≤n are i.i.d sub-gaussian random variable with mean 0 and variance σ 2 and independent of {X i· } 1≤i≤n . The explained variance under the regression model (1) is represented by the weighted quadratic functional of β, We study estimation and inference for the explained variance in the semi-supervised setting, where the data is a combination of the labelled data {y i , X i· } 1≤i≤n in the regression model (1) and the unlabelled data {X i· } n+1≤i≤n+N . Here the measured covariates of both the labelled and unlabelled data are assumed to be independent and follow the same distribution. The more conventional supervised setting is treated as a special case. The setting of semi-supervised learning is commonly seen in applications where the outcomes are more expensive to collect than the covariates. For example, in the analysis of Electronic Health Records (EHR) databases, the covariates are easy to be automatically extracted while labelling of the outcomes is costly and timeconsuming [Chakrabortty andCai, 2017, Gronsbell andCai, 2017]. In addition, semi-supervised learning naturally arises in the integrative analysis of multiple (genetics) data sets where the covariates are the same across all data sets but the outcomes measured vary from study to study due to the specific purposes of individual studies [van Iperen et al., 2017]. This can be naturally formulated as semi-supervised learning, where the pre-specified outcome is only measured over one or several (but not all) data sets while the covariates are measured across all data sets. See Chakrabortty and Cai [2017], Gronsbell and Cai [2017], Azriel et al. [2016], Zhang et al. [2016] for more discussion about semi-supervised learning.
The development of the optimal estimator and confidence intervals for Q = β ⊺ Σβ in the semi-supervised setting along with the corresponding statistical analysis is of significant interest on its own right and poses many challenges. This inference problem is also closely connected to several other important statistical problems.
(a) Heritability. Heritability is among the most important genetics concepts.
High-dimensional linear regression (1) has proven to be useful in modeling the phenotype-genotype relationship in the presence of the large amount of genetic variants [Owen, 2012, Guo et al., 2017b, Verzelen and Gassiat, 2016, Janson et al., 2015. Under the linear model (1) with the outcome normalized to have unit variance, one heritability measure defined in the literature is the quadratic functional, β ⊺ Σβ, which measures the total variance explained by genetic variants [Janson et al., 2015, Verzelen andGassiat, 2016].
(b) Signal-to-Noise Ratio and Proportion of Variance Explained. Proportion of Variance Explained (PVE) and Signal-to-Noise Ratio (SNR) are important statistics concepts and are defined respectively as β ⊺ Σβ/(β ⊺ Σβ + σ 2 ) and β ⊺ Σβ/σ 2 under model (1). The quadratic functional β ⊺ Σβ is central to SNR and PVE. Together with a good estimator of σ 2 [Antoniadis and Fan, 2001, Sun and Zhang, 2012, Belloni et al., 2011, the results for β ⊺ Σβ established in this paper are useful for inference of SNR and PVE.
(c) Signal Detection and Global Testing. Inference for the explained variance can be applied to testing the global hypothesis H 0 : β = β null for β null ∈ R p , which includes signal detection as a special case with β null = 0. The connection is revealed in the following adjusted linear model, Under model (3), testing for H 0 : β = β null is recast as testing the hypotheses H 0 : (d) Prediction Accuracy Assessment. Accuracy assessment is of significant importance in applications. Inference for the explained variance is useful for assessing the out-of-sample prediction accuracy of a given estimator. We use (X (0) , y (0) ) to denote the set of training observations andβ to denote a given estimator of β based on the training data set (X (0) , y (0) ); we use (X, y) to denote the set of test observations. The prediction accuracy for a future observation x new is defined as . To obtain the inference results for this quantity, we rely on the following adjusted linear model, Inference results developed for the explained variance can be applied to (4) to obtain the corresponding results for the prediction accuracy E xnew ( x ⊺ new (β − β) ) 2 .
(e) Confidence Ball for β. Construction of confidence balls for β is another important application of inference for explained variance studied in this paper. We useβ to denote a pre-specified estimator of β, which serves as the center of the constructed confidence ball. Based on the adjusted linear model (4), a confidence interval (L(Z), U (Z)) for the explained variance (β − β) ⊺ Σ(β − β) leads to a natural confidence ball for β, where λ min (Σ) denotes the smallest eigenvalue of Σ.
The close connections to the above statistical applications provide further motivations for studying the inference problem for β ⊺ Σβ. In Section 4, we demonstrate in detail how to apply the obtained results for β ⊺ Σβ to tackle some of these statistical applications.

Results and Contributions
We introduce a new estimator, Calibrated High-dimensional Inference for Variance Explained (CHIVE), in the semi-supervised setting. The CHIVE estimator for Q = β ⊺ Σβ is constructed in two steps, which together efficiently integrate both labelled and unlabelled data. The first step is to plug in the estimators of β and Σ, denoted by β and Σ, respectively, and the second step is to calibrate this plugin estimator β ⊺ Σ β through estimating its estimation error. The second step is essential in rebalancing the bias and variance to improve the estimation accuracy. The calibration technique is a general machinery as it can take different forms of β and Σ as its inputs. This flexibility is quite useful in the semi-supervised setting, where the unlabelled data can be efficiently used to estimate the design covariance matrix Σ. We show the optimality of CHIVE by establishing the minimax optimal rate of convergence for estimating β ⊺ Σβ in the general semi-supervised setting. We also quantify the uncertainty of the CHIVE estimator by establishing its limiting distribution under stronger conditions. Data-driven confidence intervals for β ⊺ Σβ are constructed based on the limiting distribution. The supervised setting and the setting with known design covariance matrix are discussed as the special cases with N = 0 and N = ∞, respectively. We further develop a randomized calibration technique for statistical inference in the presence of weak signals and apply the obtained results to several important statistical problems. The numerical performance of the proposed methodology is demonstrated in simulation studies and a real data analysis of estimating heritability for a yeast segregant data set with multiple traits.
The main contributions of the present paper are three-fold.
(a) We propose a novel estimator, CHIVE, for the explained variance β ⊺ Σβ that efficiently uses both the labelled and unlabelled data and is shown to achieve the minimax optimal rate. The results characterize how the unlabelled data affects the minimax optimal rate for estimating β ⊺ Σβ. Specifically, the optimal rate is ∥β∥ 2 / √ n + ∥β∥ 2 2 / √ N + n + k log p/n, where p is the dimension, n is the size of the labelled data, N is the size of the unlabelled data, and k and ∥β∥ 2 denote respectively the sparsity and the ℓ 2 norm of β. It is interesting to note that the unlabelled data only helps reduce the convergence rate ∥β∥ 2 2 / √ N + n but not the other two terms.
(b) We quantify the uncertainty for the CHIVE estimator through establishing its limiting distribution. It is shown that the limiting distribution is normal and its variance depends on the proportion of the labelled data. The result is then used for the construction of data-driven confidence intervals for β ⊺ Σβ.
(c) The inference results obtained in this paper are applied to (i) signal detection and global testing, (ii) prediction accuracy evaluation, and (iii) confidence ball construction. For signal detection, we control the type I error and characterize the type II error by establishing the power function under a local alternative. The results can be easily extended to the general global testing problem. For evaluation of out-of-sample prediction accuracy of a given sparse estimator, both the point and interval estimators are developed. We establish the estimation error bound for the point estimator of the prediction accuracy and control the length of the corresponding confidence interval. A confidence ball for the regression vector β with controlled radius is also constructed.
We stress that these procedures are data-driven and do not require a priori knowledge of the design covariance matrix Σ or the noise level σ. See more details in Section 4 and the related numerical performance in Sections 5.2 and 5.3.
A central question in semi-supervised learning is how to efficiently use both labelled and unlabelled data to conduct statistical inference [Chakrabortty andCai, 2017, Gronsbell andCai, 2017]. The results obtained in the present paper illustrate how the unlabelled data can facilitate statistical inference for the explained variance and also the related statistical applications.

Related Work
Estimation and inference for quadratic functionals have been studied in the literature in a range of settings. In particular, minimax and adaptive estimation of quadratic functionals plays an important role in nonparametric inference and has been well studied in density estimation, nonparametric regression, and white noise with drift model. See, for example, Bickel and Ritov [1988], Donoho and Nussbaum [1990], Efromovich and Low [1996], Laurent and Massart [2000], Cai andLow [2005, 2006], Collier et al. [2015].
In high-dimensional linear regression, estimation and inference for quadratic functionals has also been studied in Janson et al. [2015], Verzelen and Gassiat [2016], Guo et al. [2017b]. In particular, Verzelen and Gassiat [2016] and Guo et al. [2017b] considered estimation of β ⊺ Σβ/σ 2 and ∥β∥ 2 2 , respectively, but not the uncertainty quantification problem. Janson et al. [2015] studied the construction of confidence intervals for ∥β∥ 2 2 under the setting of Σ = I, moderate dimension where n/p → ξ ∈ (0, 1) and no sparsity assumption on β. The inference problem in sparse high-dimensional linear regression considered in the current paper is significantly different from the setting considered in Janson et al. [2015], mainly due to the complicated geometry induced by the sparsity structure and the unknown design covariance matrix Σ. Other works related to quadratic functional inference include construction of confidence intervals for the ℓ 2 loss of the estimator considered in Cai and Guo [2017a] and inference for treatment effect and endogeneity parameter in instrumental variable regression Guo et al. [2016Guo et al. [ , 2017a. In addition, Javanmard and Lee [2017], Zhu and Bradic [2017] considered hypothesis testing for high-dimensional linear regression.
The statistical applications studied in this paper have also been considered separately in the literature. Signal detection was studied in Ingster et al. [2010], Arias-Castro et al. [2011] under the linear model (1) in a special setting where the design covariance matrix Σ is equal to or closed to the identity matrix. In this setting, Ingster et al. [2010], Arias-Castro et al. [2011] established optimal signal detection method and theory. The obtained inference results in the present paper enable the study of the signal detection problem under a general setting where the design covariance matrix Σ is unknown. The confidence ball construction for the whole regression vector was considered in Nickl and van de Geer [2013] in the case of known σ and the optimal size and possibility of adaptive confidence balls was also established. The results obtained in the current paper lead to a confidence ball construction for β in the case of unknown σ. A problem related to prediction accuracy is inference for the estimation accuracy, which was considered in Cai and Guo [2017a], Janson et al. [2015]. However, inference for the prediction accuracy and that for the estimation accuracy are quite different problems.

Organization of the Paper
The rest of the paper is organized as follows. In Section 2, we introduce in detail the CHIVE estimator and establish its minimax rate optimality. Section 3 focuses on quantifying the uncertainty of the CHIVE estimator and construction of confidence intervals for β ⊺ Σβ. We apply in Section 4 the developed procedures to tackle three important problems, signal detection and global testing, prediction accuracy evaluation and confidence ball construction. Simulation results are given in Section 5 and an analysis of a yeast data set is presented in Section 6. A discussion is provided in Section 7 and the proofs are given in Section 8. Additional proofs and simulation results are presented in the appendix.

Optimal Estimation of β ⊺ Σβ
In this section, we first introduce the calibration methodology for estimating the explained variance and then establish the minimax convergence rate for estimating β ⊺ Σβ in the general semi-supervised framework. The results demonstrate the effect of the unlabelled data on the optimal convergence rate. The supervised setting and the setting with known design covariance matrix are then discussed as special cases. We begin with the notation that will be used in the rest of the paper.
We use Z = (X, y) to denote the data set. For a matrix A, A i· , A ·j , and A i,j denote respectively the i-th row, j-th column, and (i, j) entry of the matrix A. The spectral norm of A is ∥A∥ 2 = sup ∥x∥2=1 ∥Ax∥ 2 . For a symmetric matrix A, λ min (A) and λ max (A) denote respectively the smallest and largest eigenvalue of A. For a set S, |S| denotes the cardinality of S. For a vector x ∈ R p , supp(x) denotes the support of x and the ℓ q norm of x is defined as ∥x∥ q = ( 1 q for q ≥ 0 with ∥x∥ 0 = |supp(x)| and ∥x∥ ∞ = max 1≤j≤p |x j |. For a ∈ R, a + = max {a, 0}. We use c and C to denote generic positive constants that may vary from place to place. For a sequence of random variables X n indexed by n, we use X n p → X to represent that X n converges to X in probability. For a sequence of random variables X n and numbers a n , we define X n = o p (a n ) if X n /a n converges to zero in probability. For two positive sequences a n and b n , a n ≲ b n means a n ≤ Cb n for all n and a n ≳ b n if b n ≲ a n and a n ≍ b n if a n ≲ b n and b n ≲ a n , and a n ≪ b n if lim n→∞ a n b n = 0 and a n ≫ b n if b n ≪ a n .

Calibration of Plug-in Estimators
In the semi-supervised setting, the data is mixed of the labelled data (X 1· , y 1 ), · · · , (X n· , y n ) and the unlabelled data X n+1· , , · · · , X n+N · , where X 1· , · · · , X n· , X n+1· , , · · · , X n+N · are i.i.d realizations of p-dimensional covariates. We use β and Σ to denote certain "reasonably good" estimators of β and Σ, which will be specified later. Based on β and Σ, a natural estimator of the quadratic functional Q = β ⊺ Σβ is the plug-in estimator β ⊺ Σ β, which has the following error decomposition, Based on the above decomposition, the estimator β ⊺ Σ β can be further improved since the estimation error due to the first term 2 β ⊺ Σ( β − β) on the right hand side of (6) can be further reduced. We estimate the term 2 β ⊺ Σ( β − β) in the error decomposition (6) by −2 β ⊺ 1 n ∑ n i=1 X i· (y i − X i· β) and propose the following calibrated estimator, This estimator is referred to as the CHIVE estimator, as a shorthand for Calibrated High-dimensional Inference for Variance Explained. The calibration step in (7) is essentially to improve the plug-in estimator β ⊺ Σ β through re-balancing the bias and variance. The calibrated estimator requires three inputs, the initial estimators β and Σ and the data Z = (X, y). With this machinery, it remains to propose initial estimators for β and Σ. We begin with estimators for β and then move on to the estimators for Σ. Throughout the paper, without special notification, we make the following assumptions on the estimators β and σ 2 .
(B1) With probability larger than 1−γ(n) where γ(n) → 0, the estimator β satisfies (B2) σ 2 is a consistent estimator of σ 2 , that is, σ 2 σ 2 − 1 Examples of estimators satisfying (B1) and (B2). The scaled lasso estimator { β, σ 2 } defined in the following equation (8) has been shown in Sun and Zhang [2012] to satisfy (B1) and (B2) under regularity conditions, See also Lemma 1 in Guo et al. [2017b] for more details. Since the square root lasso estimator [Belloni et al., 2011] is numerically the same with the scale Lasso estimator, the square root lasso estimators of β and σ also satisfy (B1) and (B2). In addition, with a prior knowledge of σ, the Lasso estimator of β and other variants are also shown to satisfy the above condition (B1); see Candès and Tao [2007], Zhang [2010], Ye and Zhang [2010] for more details. Now, we turn to the estimators of Σ. The additional unlabelled data is useful for estimating the design covariance matrix Σ. We pool the information contained in both the labelled and unlabelled data and estimate Σ by Then we use β and Σ S as inputs and utilize the calibration idea introduced in (7), (9) When there is no confusion, we use Q to denote the estimator proposed in (9). We first introduce the following regularity conditions and then establish the convergence rate of the proposed estimator in (9) in Theorem 1.
(A1) The rows X i· are i.i.d. p-dimensional sub-gaussian random vectors with mean 0 and covariance matrix Σ with 1/M 1 ≤ λ min (Σ) ≤ λ max (Σ) ≤ M 1 for M 1 ≥ 1; The errors {ϵ i } 1≤i≤n are independent of {X i· } 1≤i≤n+N and follow i.i.d sub-gaussian random variable with mean zero and variance σ 2 ; The highdimensional vector β is assumed to be of sparsity k; (A2) There exists some positive constant c 0 > 0 such that E Assumption (A1) requires that the spectrum of the covariance matrices Σ is bounded away from zero and infinity and that the noise level σ is upper bounded by a constant. Assumption (A1) also assumes that both the design and the noise are subgaussian. Define U = X ⊺ i· β/ √ β ⊺ Σβ, where E(U ) = 0 and E(U 2 ) = 1. Assumption (A2) is placed on this random variable U such that Var(U 2 ) is not vanishing. This assumption is imposed such that Var(U 2 ) can be well estimated and this type of assumption has been introduced in covariance matrix estimation literature [Cai and Liu, 2011] for the same purpose.
Theorem 1. Suppose that Condition (A1) holds and k ≤ cn/ log p for some constant c > 0. For any estimator β satisfying Condition (B1), with probability Under the additional assumption k ≪ √ n/log p and ∥β∥ 2 ≫ k log p/ √ n, where ρ = lim n→∞ n N +n .
Remark 1. Since Q ≥ 0, the convergence rate (10) also holds for Q + , the positive part of Q. To keep the notation simpler, we only present the results for Q throughout this paper. This convergence rate established in (10) is shown to be optimal in Section 2.2. Under the additional assumptions k ≪ √ n/log p and ∥β∥ 2 ≫ k log p/ √ n, we establish a more refined distributional result in (11). Such normal limiting distribution is used in Section 3 to construct confidence intervals for β ⊺ Σβ. One interesting phenomenon is that the limiting distribution established in (11) depends on the proportion of the labelled data. If the amount of unlabelled data dominates that of labelled data (that is, ρ = 0), then the limiting distribution in (11) is simplified to √ n( Q−Q) √ 4σ 2 β ⊺ Σβ → N (0, 1).

Optimal Rate of Convergence
In this section, we further investigate the optimality of the proposed estimator (9) by studying the minimax convergence rate of estimating β ⊺ Σβ in the semi-supervised setting and consider the following specific parameter space, where M 1 ≥ 1 and M 2 > 0 are positive constants. The parameter space defined in (12) requires the sparsity ∥β∥ 0 ≤ k and M/2 ≤ ∥β∥ 2 ≤ M , where k and M are allowed to grow with n and p. Here k quantifies the sparsity of β and M quantifies the signal strength of the true signal β in terms of its ℓ 2 norm. The other conditions 1/M 1 ≤ λ min (Σ) ≤ λ max (Σ) ≤ M 1 and σ ≤ M 2 are regularity conditions. The following theorem establishes the minimax lower bounds for the convergence rate of estimating Q over the parameter space Θ(k, M ).
Theorem 2. Suppose k ≤ c min {n/ log p, p ν } for some constants c > 0 and 0 ≤ ν < 1 2 . Then In the above theorem, only the first term in the lower bound is involved with the amount of the additional unlabelled data, that is to say, a larger amount of unlabelled data only helps lower the term M 2 / √ N + n but not any other terms. Theorems 1 and 2 together show that the estimator proposed in Section 2.1 is minimax rate optimal under regularity conditions. Corollary 1. Suppose that Condition (A1) holds and k ≤ c min {n/ log p, p ν } for some constants c > 0 and 0 ≤ ν < 1 2 . For any estimator β satisfying Condition (B1), the estimator Q defined in (9) is minimax rate optimal over Θ(k, M ) where √ k log p/n ≲ M ≤ C for some constant C > 0.
The above corollary shows that the proposed method attains the optimal convergence rate when the ℓ 2 norm is relatively strong, that is, ∥β∥ 2 is bounded away from zero by √ k log p/n. As shown in Theorem 2, for the case where M ≪ √ k log p/n, the lower bound of estimating β ⊺ Σβ is M 2 . This optimal convergence rate can be achieved by a trivial estimator 0 and hence the corresponding regime M ≪ √ k log p/n is not interesting in terms of studying optimal estimators. In Corollary 1, the lower bound (13) is only matched for the regime where M ≤ C for some constant C > 0. For theoretical interest, we are going to modify the proposed estimator Q defined in (9) such that the modified version will achieve the lower bound (13) over the whole interesting regime M ≳ √ k log p/n. We randomly split the data (y, X) into two subsamples Z (1) = ( y (1) , X (1) ) with sample size n 1 and Z (2) = ( y (2) , X (2) ) with sample size n 2 , where n 1 ≍ n 2 . Let β denote an estimator which is produced by the first sub-sample ( y (1) , X (1) ) and satisfies Condition (A1). One example of such an estimator is the scaled Lasso estimator (8) applied to the subsample Z (1) = ( y (1) , X (1) ) . We propose the following estimator of Q, where The following theorem establishes the convergence rate of Q( β, Σ (2) , Z (2) ) and shows that this estimator achieves the optimal convergence rate of estimating Q for M ≳ √ k log p/n.

Two Special Cases
We now turn to the inference in the supervised setting and the setting with known design covariance matrix. These two settings can be viewed as the special cases with N = 0 and N = ∞ respectively.

Case I: Supervised Inference
In the supervised setting, we only observe the labelled data and will estimate Σ by the sample covariance matrix The following theorem establishes the convergence rate of the estimator Q = Q( β, Σ L , Z).
Theorem 4. Suppose that Condition (A1) holds and k ≤ cn/ log p for some constant c > 0. For any estimator β satisfying (B1), with probability larger than Under the additional assumption (A2) and ∥β∥ 2 ≫ min The estimator Q( β, Σ L , Z) is a special case of the estimator (9) with N = 0. By comparing Theorem 4 with Theorem 1, if ∥β∥ 2 ≤ C for some positive constant C, then the unlabelled data leads to a faster convergence rate by reducing the term ∥β∥ 2 2 / √ n in (17) to ∥β∥ 2 2 / √ N + n in (10); however, the unlabelled data does not affect other terms in the convergence rate. The effect of the unlabelled data is also revealed in the limiting distribution of the proposed estimator, where a comparison of (18) and (11) shows that the exact variance level is reduced from where ρ denotes the limiting proportion of the amount of labelled data out of the total amount of both labelled and unlabelled data. The following corollary further establishes the minimax rate for estimating β ⊺ Σβ in the supervised setting.
Corollary 2. Suppose that Condition (A1) holds and k ≤ c min {n/ log p, p ν } for some constants c > 0 and 0 ≤ ν < 1 2 . For any estimator β satisfying Condi- achieves the following optimal estimation rate over Θ(k, M ) for M ≳ √ k log p/n, Remark 2. A related paper [Guo et al., 2017b] studies estimation of ∥β∥ 2 2 and shows that the optimal rate of estimating ∥β∥ 2 2 over Θ(k, M ) for M ≳ √ k log p/n is M/ √ n + (M + 1)k log p/n in the supervised setting. In contrast to (19), we can see that neither of these two problems is easier than the other, where there is an additional term M 2 / √ n in (19) and an additional term M k log p/n in the optimal convergence rate of estimating ∥β∥ 2 2 . Remark 3. Inference for β ⊺ Σβ is closely connected to Sun and Zhang [2012], Verzelen and Gassiat [2016], where Sun and Zhang [2012] studied the inference problem for σ 2 and Verzelen and Gassiat [2016] studied the estimation of β ⊺ Σβ/σ 2 . In particular, Sun and Zhang [2012] proposed the scaled lasso estimator σ 2 in (8) to estimate σ 2 and Verzelen and Gassiat [2016] proposed to estimate β ⊺ Σβ by (7), if β is taken as the scaled Lasso estimator, then Q( β, Σ L , Z) is reduced to being the same as the estimator proposed in Verzelen and Gassiat [2016], where the equivalence is shown by the following expression, We shall stress that the calibration idea in (7) provides a completely new perspective on estimation of β ⊺ Σβ, where instead of using the expression Q = E(y 2 i ) − σ 2 and estimating σ 2 first, we estimate Q directly by calibrating the plug-in estimator. This new perspective establishes a general machinery taking reasonable good initial estimators of β and Σ as inputs. As shown in (9), the flexibility of the calibrated estimator has proven to be extremely useful in efficiently pooling additional information on Σ; Note that the estimation method introduced in Verzelen and Gassiat [2016] cannot be extended to the semi-supervised setting as that for the calibration perspective in (9). Additionally, Verzelen and Gassiat [2016] focused on the estimation problem instead of confidence interval construction and hypothesis testing problems. In terms of technical details on estimation optimality, the results in Verzelen and Gassiat [2016] allowed for a more general regime k ≥ √ p than Corollary 2 but only considered the optimality in the supervised setting and considered a fixed M in the analysis.

Case II: Known Σ
The general semi-supervised results also shed light on another interesting setting where the design covariance Σ is known. In the semi-supervised setting, the additional unlabelled data is used for estimating the design covariance matrix Σ. The case of known Σ is an extreme case of the semi-supervised setting with N taken as infinity. The estimator (14) can be modified such that the information on Σ is incorporated, Similarly, the estimator proposed in (9) can be modified as Corollary 3. Suppose that Condition (A1) holds and k ≤ cn/ log p for some constant c > 0.
(a) For any estimator β depending on the first half sample ( y (1) , X (1) ) and satisfying Condition (B1), with probability larger than (b) For any estimator β satisfying Condition (B1), with probability larger than Through comparing (23) with (15) and (24) with (10), the uncertainty of estimating the design covariance matrix leads to the additional term ∥β∥ 2 2 / √ N + n. By applying Theorem 2, we can show that the convergence rate in (23) achieves the optimal convergence rate M/ √ n + k log p/n. The term M 2 / √ N + n will disappear due to the known design covariance matrix Σ.

Confidence Intervals for β ⊺ Σβ
In this section, we consider the problem of constructing confidence intervals for β ⊺ Σβ, which is involved with uncertainty quantification of the CHIVE estimator proposed in Section 2. We first construct confidence intervals for β ⊺ Σβ in Section 3.1 and introduce a randomized calibration procedure in Section 3.2 to study inference for explained variance in the presence of weak signals.

Confidence Interval Construction
We start with uncertainty quantification of the CHIVE estimator Q proposed in (9). By the limiting distribution established in (11), the main next step is to consistently estimate the standard error with Σ S defined in (9). Then we propose the following confidence interval centered at Q, where z α/2 is the upper α/2 quantile of standard normal distribution. The following theorem establishes the coverage and precision properties of CI(Z), where the length of any interval CI Theorem 5. Suppose that Conditions (A1) and (A2) hold, k ≪ min{n/(log(N + n) log p), √ n/log p} and ∥β∥ 2 ≫ k log p/ √ n. For β and σ 2 satisfying Conditions (B1) and (B2), respectively, the confidence interval given in (25) satisfies the following coverage and precision properties, for any positive constant δ 0 > 0.
The effect of the additional data on the length of confidence interval is demonstrated in (27), where the confidence interval gets shorter with a larger amount of unlabelled data. Furthermore, the length which matches the optimal convergence rate of estimation M/ √ n + M 2 / √ N + n over the parameter space Θ(k, M ) for k ≪ √ n/log p and M ≫ k log p/ √ n. As shown in Theorem 5, the validity of the proposed confidence interval (25) requires the condition that ∥β∥ 2 is bounded away from zero by k log p/ √ n. Although k log p/ √ n converges to zero over the extreme sparse regime k ≪ √ n/log p, it reveals the difficulty of constructing stable confidence intervals for β ⊺ Σβ when ∥β∥ 2 is at a local neighborhood of zero. The next section will address the inference problem in presence of such weak signals.

Inference for Weak Signals: Randomized Calibration
As discussed in the introduction, uncertainty quantification of Q = β ⊺ Σβ is closely connected to other important statistical problems, including (1) signal detection and global testing; (2) prediction accuracy evaluation and (3) confidence ball construction. These applications provide a strong motivation for inference for the explained variance under the settings of weak signals (that is, ∥β∥ 2 ≲ k log p/ √ n). In the following, we focus on the inference problem in the presence of weak signals and introduce a randomized version of the CHIVE estimator (9). We first generate ran- which is independent of the observed data Z. Similar to (9), we propose the following randomized calibrated estimator, When there is no confusion, we use Q R to denote the estimator proposed in (28).
In contrast to (9), the calibration step in (28) is involved with an additional term 2 1 If u i is zero instead of being generated as normal random variables in (28) is reduced to being exactly the same as Q (9). Since u i in (28) is randomly generated normal random variables, this additional term approximately follows a normal distribution with mean zero and variance 4σ 2 τ 2 0 /n. Even in the presence of weak signals, this additional term further enlarges the variance level of the calibrated estimator such that the bias level of the calibrated estimator is dominated by the corresponding variance level. The following corollary establishes the limiting distribution of the estimator Q R after randomized calibration.
Theorem 6. Suppose that Condition (A1) holds, k ≪ √ n/ log p and τ 0 > 0 is a positive constant. For any estimator β satisfying Condition (B1), then where ρ = lim n n+N . In comparison to the limiting distribution (11) in Theorem 1, Theorem 6 requires no condition on ∥β∥ 2 to conduct inference for the explained variance while the variance level of Q R is slightly larger than that of Q by the amount 4σ 2 τ 2 0 /n. This additional variance term is a side effect of the randomized calibration. However, it has paved the way to quantify the uncertainty when ∥β∥ 2 is near zero (that is ∥β∥ 2 ≲ k log p/ √ n). Similar to (25), the standard error of the randomized estimator Q R in (29) is approximated as where Σ S is defined in (9). Then we propose the following confidence interval, where z α/2 is the upper α/2 quantile of standard normal distribution. The following corollary characterizes the coverage and precision properties of CI R (Z).
Corollary 4. Suppose that Conditions (A1) and (A2) hold, k ≪ min{n/(log(N + n) log p), √ n/log p} and τ 0 > 0 is a positive constant. For β and σ 2 satisfying Conditions (B1) and (B2), respectively, then the confidence interval defined in (31) satisfies the following coverage and precision properties, and The algorithm for estimating β ⊺ Σβ and quantifying its uncertainty is summarized in Algorithm 1.

Algorithm 1: Randomized CHIVE in the Semi-supervised Setting
Input : Labelled data {y i , X i· } 1≤i≤n and unlabelled covariates {X i· } n+1≤i≤n+N ; Randomization level τ 0 Output: Point estimator Q R = Q R (y, X, τ 0 ) and its standard error estimator ϕ R = ϕ R (y, X, τ 0 ) 1 Initialization: Construct point estimator β and σ 2 satisfying (B1) and (B2); Estimate Σ by Σ S defined in (9); 2 Randomized Calibration: Estimate Q by the estimator Q R defined in (28), where the variables {u i } 1≤i≤n are generated to be independent of the observed data (X, y) and following i.i.d N (0, τ 2 0 ); 3 Uncertainty Quantification: Estimate the standard error of the proposed estimator by ϕ R defined in (30).

Statistical Applications
In this section, we apply Algorithm 1 to tackle several important statistical problems, including signal detection and global testing in Section 4.1, prediction accuracy evaluation in Section 4.2 and confidence ball construction in Section 4.3.

Application 1: Signal Detection and Global Testing
Signal detection is of great importance in statistics and related scientific applications and the detection problem in high-dimensional linear regression was studied in Arias-Castro et al. [2011], Ingster et al. [2010]. The inference procedure stated in Algorithm 1 has profound implications on signal detection and the general global testing in high-dimensional linear regression. We consider the global hypothesis H 0 : β = β null , which includes the signal detection as a special case with taking β null = 0. The global testing problem is cast as We apply Algorithm 1 with a given τ 0 > 0 and obtain the point estimator Q R (y − Xβ null , X, τ 0 ) and its standard error estimator ϕ R (y − Xβ null , X, τ 0 ). Then we propose the detection procedure, with Type I error controlled at α ∈ (0, 1) as We define the corresponding null parameter space as and the local alternative parameter space as The following corollary establishes that D(τ 0 ) controls the type I error asymptotically and also establishes the asymptotic power function of the proposed test.
Corollary 5. Suppose that Conditions (A1) and (A2) hold, τ 0 > 0 is a positive constant and the vector δ = β − β null satisfies the conditions that ∥δ∥ 0 ≪ min{n/(log(N + n) log p), √ n/log p} and E For ρ > 0 and any θ ∈ H 1 (∆) with some positive constant ∆ > 0, then The assumptions of Corollary 5 are the same as those of Corollary 4 from the perspective that the conditions imposed on β in Corollary 4 are now imposed on the difference vector δ = β − β null . One sufficient condition for the difference vector δ being sparse is that both the true signal β and the null hypothesis β null are sparse. Corollary 5 shows that for any positive constant τ 0 , D(τ 0 ) controls the type I error asymptotically. For the finite sample performance, we have investigated how to choose the randomization level τ 0 in the simulation section. See Section 5.2 for the numerical performance.

Application 2: Prediction Accuracy Assessment
Inference for explained variance has important applications to evaluating the out-ofsample prediction for a given sparse estimatorβ. To keep the notation consistent, we assumeβ is estimated based on a training data set (X 0 , y 0 ) and (X, y) is an independent test data to evaluate its prediction accuracy. We start with computing the residual on the test data set The out-of-sample prediction accuracy is defined as PA and it is reduced to the explained variance for the residual model (40) with outcome r = y − Xβ and covariates X. Let Q R (r, X, τ 0 ) and ϕ R (r, X, τ 0 ) denote the outputs of Algorithm 1 with the labeled data {(r i , X i· )} 1≤i≤n and unlabelled data {X i· } n+1≤i≤n+N as inputs. Then we propose the point estimator of PA(β) as Q R (r, X, τ 0 ) and the interval estimator for PA(β) as ) (41) The following corollary establishes the convergence rate for the point estimator and the coverage and precision properties of the interval estimator.
Corollary 6. Suppose that Conditions (A1) and (A2) hold and τ 0 > 0 is a positive constant. For any sparse estimator satisfying ∥β∥ 0 ≤ C∥β∥ 0 and C > 0, then (a) If k ≤ cn/ log p for some positive constant c > 0, then with probability larger some positive constant c 0 where δ = β −β, then the confidence interval defined in (41) satisfies the following coverage and precision properties, and for some constant C > 0.
The above corollary has shown that the precision of confidence interval for the prediction accuracy is not just related to the sample sizes n, N , the sparsity k and the dimension p, but also related to the accuracy of the evaluated estimator ∥β−β∥ 2 . See Section 5.3 for the numerical performance.

Application 3: Confidence Ball Construction
The prediction accuracy evaluation established in (41) can be used to construct confidence ball for β. For the setting where λ min (Σ) is known, then we have λ min (Σ)∥β − β∥ 2 2 ≤ (β − β) ⊺ Σ(β − β) and construct the confidence ball for β as As shown in (44), the radius of the confidence ball CB(β) is upper bounded by To minimize the radius, we need to select the centerβ for the confidence ball in (45) such thatβ is sparse and ∥β − β∥ 2 is small. In the high-dimensional literature, several penalized estimators are shown to satisfy such properties, such as Lasso, scaled Lasso and Dantzig Selector.

Simulation Study
We carry out simulation studies in this section to demonstrate the numerical performance of the proposed methods, which consist of confidence interval construction for β ⊺ Σβ in Section 5.1, signal detection in Section 5.2 and prediction accuracy evaluation in Section 5.3. Throughout all simulation studies in this paper, we generate the high-dimensional linear regression (1) with the dimension p = 800 and the corresponding sample size n across {200, 400, 600, 800, 1000, 1200, 2400}. For the linear model (1), the covariates {X i· } 1≤i≤n are generated in i.i.d. fashion to follow multivariate normal distribution with mean zero and covariance matrix Σ ∈ R 800×800 where Σ ij = 0.5 |i−j| and the errors {ϵ i } 1≤i≤n are generated as i.i.d standard normal distribution. In addition to the labelled data, we also generate the unlabelled data {X i· } n+1≤i≤n+N with N = 2, 000 to study the proposed inference procedures in the semi-supervised setting. The simulations are replicated over 500 simulations.
We have compared the estimation accuracy across two different types of estimators, the plug-in estimator and the CHIVE estimator and across two different settings, supervised setting and semi-supervised setting. Recall that only labelled data is available in the supervised setting and both the labelled and unlabelled data are available in the semi-supervised setting. The numerical comparison has been reported in Figure 1. Across all three settings, it is observed that the proposed CHIVE estimator has achieved uniformly much better estimation accuracy than the plug-in estimators, in both supervised and semi-supervised settings. This numerical observation demonstrates that the calibration step is useful in improving the estimation accuracy. In addition, the unlabelled data is useful in estimating β ⊺ Σβ, where as demonstrated in Figure 1, the solid line (the CHIVE estimator in the semi-supervised setting) is always below the dotted line (the CHIVE estimator in the supervised setting). This matches with the theoretical results.  In addition to the significant improvement in terms of estimation, the CHIVE estimator serves as the center of confidence intervals for β ⊺ Σβ. The coverage and precision properties of the constructed confidence interval CI are reported in Table 1. With a larger sample size, the empirical coverage of the proposed confidence interval achieves 95% and the average lengths of the confidence intervals get shorter. The integration of the unlabelled data in the semi-supervised setting has shorten the lengths of confidence interval significantly. for the given setting. Each row reports empirical coverage (indexed with "Cov") and average lengths (indexed with "Len") of proposed CIs. The columns indexed with "Supervised" represent the results for the supervised setting and the columns indexed with "Semi-Supervised" represent the results for the semi-supervised setting. For example, in the first row of numbers (0.922, 3.750, 0.896, 1.796), it corresponds to the setting 1 and sample size n = 200, in the supervised setting, CI has empirical coverage 0.922 and the average length is 3.750; in the semi-supervised setting, CI has empirical coverage 0.896 and the average length is 1.796.

Signal Detection
For the detection problem, we generate β as β j = δ for 1 ≤ j ≤ 50 and β j = 0 for 51 ≤ j ≤ 800 and vary δ across {0.00, 0.025, 0, 05, 0.075, 010, 0.125, 0.15} and vary the sample size n across {600, 1200}. In Figure 2, we demonstrate the coverage and precision properties of the randomized confidence intervals across four methods, the non-randomized detector D(0) and the three randomized detectors D(2), D(4) and D (6), where D(·) is defined in (35). The two plots on the top of Figure 2, corresponding to the supervised setting with n = 600 demonstrate the effect of randomization on the empirical coverage and average lengths, where the randomization leads to a interval estimator achieving the coverage properties at the expense of wider interval estimators. With the randomization level τ 0 reaching 2, the coverage property is guaranteed while the empirical coverage for the procedure without randomization (τ 0 = 0) is much lower than 0.95, especially for weak signals with a small δ. The bottom two plots of Figure 2 corresponds to the supervised setting with n = 1, 200 and the main observation is similar to the case of n = 600 but the confidence intervals are much shorter than the setting with n = 600. The empirical detection rate is reported in Table 2, where the sample size n is generated across n = 600 and n = 1, 200 and the explained variance β ⊺ Σβ is controlled via the scaler δ. When δ = 0, it corresponds to the null case and a proper detection procedure is expected to have type I error rate 0.05. As predicted by theory, the detection method without randomization D(0) fails to give proper type I error due to presence of weak signals. With introducing the randomization procedure, the type I error rate gets closer to 0.05. When δ moves away from zero, the detection procedure is taken as a powerful procedure as the empirical detection rate approaches 1. For the detection procedure with randomization level τ 0 = 2, the setting with δ = 0.025 corresponds to an indistinguishable region, where it is challenging to detect the signal. However, as δ reaches 0.05, the detection rate reaches 0.800 for n = 600 and 0.944 for n = 1200. As characterized by theory, a larger randomization level requires a higher value of δ such that the signal can be detected, for example, for τ 0 = 4, until δ reaches 0.075, the detection rate reaches 0.82 for n = 600 and 0.968 for n = 1200. The corresponding semi-supervised setting shows a similar phenomenon to the supervised setting but tends to be easier than the supervised setting due to the unlabelled data. The results are reported in the supplementary materials. .000 1.000 0.820 0.524 1.000 1.000 0.968 0.764 0.100 1.460 1.000 1.000 1.000 0.914 1.000 1.000 1.000 0.992 0.125 2.281 1.000 1.000 1.000 0.996 1.000 1.000 1.000 1.000 0.150 3.285 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 Table 2: Empirical detection rates in the supervised setting. The column indexed with δ represents the signal strength, where the signal is of sparsity 50 and of the form δ · (1, 1, · · · 1, 0, 0, · · · , 0); the column indexed with β ⊺ Σβ represents the value of β ⊺ Σβ; the columns under "n=600" and "n=1,200" correspond to sample size 600 and 1,200 respectively, where the column indexed with D(τ 0 ) report the empirical detection rates for the detector D(τ 0 ).

Prediction Loss Evaluation
In this subsection, the high-dimensional regression vector β is generated with sparsity 10 where β j = j/5 for 1 ≤ j ≤ 10 and β j = 0 for j ≥ 11. Let β(λ) denote the Lasso estimator based on an independent training data ( X (0) , y (0) ) with sample size n 0 = 600, We consider the inference problem for the out-of-sample prediction accuracy ( β(λ)− β) ⊺ Σ( β(λ)−β). Specifically, we consider three estimators β(λ 0 ), β(5λ 0 ) and β(10λ 0 ) and report the numerical performance of both point and interval estimators of the corresponding prediction accuracy. We consider the prediction accuracy problem across three different sample sizes, {600, 1200, 2400} and introduce different randomization levels. We will use PA(τ 0 ) to denote the procedure with randomization level τ 0 . Table 3 has reported the point and interval estimators of the prediction accuracy across different settings. In terms of point estimation, the sample averages get closer to the true accuracy with increasing sample sizes. Among the three estimators, the true prediction accuracy of β(λ 0 ) is the smallest and also the most difficult to assess. The fundamental reason is that the small accuracy/error is hard to quantify. This phenomenon is connected to the theoretical results established in Cai and Guo [2017a], which showed that the estimation accuracy ∥ β − β∥ 2 2 is hard to quantify for an accurate estimator β. The constructed confidence interval PA(0) without randomization has no coverage even for n = 2400. In such a scenario with weak signals, the evaluators involved with randomized calibration, PA(2) and PA(4) produce valid confidence intervals across different settings.
Not only the point and interval estimators are useful, the upper limit and lower limit of the confidence intervals reported in Table 3 can also be informative in the prediction accuracy evaluation. For the estimator β(λ 0 ), although the average of lower limit of confidence intervals for ( β(λ 0 ) − β) ⊺ Σ( β(λ 0 ) − β) is zero, the corresponding upper limits of confidence intervals are informative as they provide empirical guidance to practitioners with upper bounds for the prediction accuracy. For β(5λ 0 ) and β(10λ 0 ), both the upper and lower limits of the confidence intervals are informative on the size of the prediction accuracy.   The table reports six settings, corresponding to three different sample sizes (600,1200, 2400) and the supervised and semi-supervised setting. For example, "Super, 600" stands for the supervised setting with sample size n = 600 and "Semi, 600" stands for the semi-supervised setting with sample size n = 600. The true prediction accuracy of the three estimators β(λ 0 ), β(5λ 0 ) and β(λ 0 ) is reported as 0.065, 0.636 and 2.310. Three prediction accuracy evaluators PA(0), PA(2) and PA(4) are reported, where PA(0) is the evaluator with no randomization, PA(2) is the evaluator with randomization level τ 0 = 2 and PA(4) is the evaluator with randomization level τ 0 = 4. For each setting, the row indexed with "Coverage" reports the empirical coverage of the corresponding confidence intervals over 500 simulations; the row indexed with "Est Aver" reports the sample average of the corresponding point estimators over 500 simulations; the rows indexed with "Lower Aver" and 'Upper Aver" report the sample averages of the lower and upper limits of interval estimators over 500 simulations.

Real Data Application
In this section, we analyze a yeast data set reported in Bloom et al. [2013] and study how the genetic variants explain the colony sizes under different growth media. The goal is to estimate the heritability measures of colony sizes under different growth media, which represent the variance of the colony sizes explained by the genetic variants.
Bloom et al. [2013] investigated a large scale genome-wide association study of 46 quantitative traits based on 1,008 Saccharomyces cerevisiae segregants crossbred from a laboratory strain and a wine strain. These quantitative traits are measures of end-point colony size under 46 different growth media, including Hydrogen Peroxide, Cadmium Chloride, Calcium Chloride, Lactose, Raffinose, Sorbitol, Yeast Nitrogen Base (YNB) and Yeast Peptone Dextrose (YPD). The genetic maker genotypes are coded as 1 or −1, according to which strain it comes from. A set of 11,623 unique genotype markers of the 1,008 segregants is measured. Since many of these markers are highly correlated and the corresponding codes are only different in serval samples, Bloom et al. [2013] further selected a set of 4, 410 markers that are weakly dependent based on the linkage disequilibrium information. All traits are normalized to have unit variance and hence the explained variance is a measure for heritability. Bloom et al. [2013] showed that the genetic variants are associated with many of such trait values and highlighted the importance of addressing missing heritability. Bloom et al. [2013] pointed out one key reason for missing heritability as " the undiscovered factors could have effects that are too small to be detected with current sample sizes, or even too small to ever be individually detected with statistical significance". The CHIVE estimator has addressed this specific concern of missing heritability by calibrating the plug-in estimators. We demonstrate this phenomenon in Figure 3 by comparing the CHIVE estimator and the plug-in estimators of heritability for all 46 traits. We shall stress that all points lie above the line y = x and this means that the calibration step adds back the missing heritability due to simply plugging in the Lasso estimator, where the Lasso estimator tends to ignore the genetic markers with small effects.
We also construct confidence intervals for heritability of all 46 traits and report part of the results in Table 4. Note that a proportion of the outcome variables for different growth media have missing values, with the proportion of missing ranging from 0.2% to 40.58%. This forms the semi-supervised type data naturally (note that the unlabelled data is of a smaller size than the labelled data in this specific example). After applying the proposed methods to analyzing the corresponding outcomes, we have the following interesting observations, 1) the heritability measures of the colony sizes under different growth media range from 0.3 to 0.8 and all of the confidence interval estimators do not contain zero. This means the colony sizes under different growth media are strongly genetically heritable; 2) The integration of the unlabelled data has shortened the length of the constructed confidence intervals. For example the length is shorten by around 3% for Sorbitol (with 40.58% outcome missing), around 2% for Raffinose (with 34.33% outcome missing) and around 1% for Hydrogen Peroxide (with 23.71% outcome missing).

Discussions
This paper studies statistical inference for the explained variance β ⊺ Σβ in the semisupervised setting, which includes the supervised setting as a special case. By comparing the theoretical as well as the numerical results for the semi-supervised and supervised settings, it is easy to see the significant contributions of the unlabelled data to the inference accuracy. In addition, the constructed confidence interval, using the idea of calibration and randomization, has been shown to be useful in tackling other important statistical applications, including signal detection and global testing, prediction accuracy evaluation and confidence ball construction.
There remain a few open questions for future research. Although the CHIVE estimator has been shown to achieve the optimal rates over the whole sparse regime k ≲ n/log p, construction of confidence intervals for β ⊺ Σβ is only considered over the ultra-sparse regime k ≪ √ n/log p. Since both point and interval estimator do not require the prior knowledge of the exact sparsity level, they are referred to as adaptive estimation and adaptive confidence interval, respectively. However, it remains open whether it is possible to construct adaptive confidence intervals over the moderate sparse regime √ n/log p ≲ k ≲ n/log p. The possibility of adaptive confidence interval for the general linear functional η ⊺ β for η ∈ R p has been studied in Cai and Guo [2017c] and the technical tools developed in Cai and Guo [2017c] can be useful to study the adaptive confidence intervals for β ⊺ Σβ.
Due to the emerging semi-supervised data sets, it is of significant importance to propose procedures incorporating the unlabelled data efficiently and study how the unlabelled data affects the statistical accuracy. This paper has studied both methodological and theoretical perspectives of the semi-supervised statistical inference for the explained variance β ⊺ Σβ. However, it is largely unknown how these  Table 4: Confidence intervals for heritability. The column indexed with " Media" represents the growth media for the yeast segragents; The three columns under "Supervised" corresponds to the case of only using the labelled data, where the column indexed with "Plug" represents the plug-in estimator, indexed with "CHIVE" represents the CHIVE estimator, and indexed with "CI" represents the constructed confidence interval; Similarly, the three columns under "Semi-Supervised" corresponds to analyzing the semi-supervised type data, that is also using the observations with missing outcome variables. The numbers inside the parenthesis represent the standard errors of the proposed CHIVE estimators. The column indexed with "Missing" represents the proportion of missing outcome for the corresponding media.
unlabelled data can facilitate the statistical inference problem for other quantities of interests, such as the general linear functional η ⊺ β for some given η ∈ R p , quadratic functional ∥β∥ 2 2 and the variance level σ 2 . These are interesting problems left for future research.

Proof
In this section, we provide the proofs of Theorems 1 and 4. The proofs of Theorem 6, Theorem 5 and Theorem 3, Theorem 2, Corollaries 4, 5 and 6 are provided in Section A.
To establish Theorems 1 and 4, we first decompose the difference between the calibrated estimator Q = Q( β, Σ S , Z) and Q = β ⊺ Σβ, Lemma 1 characterizes the convergence rates of the last three terms in (46).
Lemma 1. Suppose that Condition (A1) holds and k ≤ cn/ log p for some constant c > 0. For any estimator β satisfying Condition (B1), then with probability larger than 1 − p −c − γ(n) − e −ct 2 , For the first two terms in (46), the following lemma establishes their convergence rate and also the limiting distribution.
Lemma 2. Suppose that Condition (A1) holds and k ≤ cn/ log p for some constant c > 0. Then with probability larger than 1 − e −ct 2 , In addition, we establish the limiting distribution Proof of Theorem 1. For the proof of Theorem 1, the convergence rate in (10) follows from the decomposition (46), Lemma 1, (50) in Lemma 2 and the fact that Under the additional assumptions k ≪ √ n/log p and ∥β∥ 2 ≫ k log p/ √ n, it follows from Lemma 1 that Combined with (51) in Lemma 2, we establish the limiting distribution (11) in Theorem 1.
Proof of Theorem 4. The proof of Theorem 4 is similar to that of Theorem 1. The main change is that Σ S in (46) is replaced by Σ L = 1 n ∑ n i=1 X i· X ⊺ i· and hence the last term 2 β ⊺ ( Σ S − 1 (46) becomes zero in this case. Hence, the convergence rate in (17) follows from the decomposition (46) and Lemma 1 and (50) in Lemma 2. Under the additional assumptions ∥β∥ 2 ≫ (51)  Yinchu Zhu and Jelena Bradic. A projection pursuit framework for testing general high-dimensional hypothesis. arXiv preprint arXiv:1705.01024, 2017.

A. Proof of Additional Theorems and Corollaries
A.1. Proof of Theorem 6 Following from (46), we establish the error decomposition of Q R − Q, (52) The theorem follows from Lemma 1, the decomposition (52) and the following lemma.

A.4. Proof of Theorem 2
We start with introducing two definitions. Define the χ 2 distance between two distributions f 1 (z) and f 0 (z) as f 0 (z) dz − 1 and the total variation distance as L 1 (f 1 , f 0 ) = ∫ |f 1 (z) − f 0 (z)| dz. It is well known that Part of the lower bound in Theorem 2 follows from the lower bounds established in Guo et al. [2017b], where, using the the current paper's terminology, equation (29) of Theorem 3 in Guo et al.
[2017b] is expressed as The constructed least favorable null and alternative hypotheses in the proof of (65) belong to the subspace θ ∈ Θ (k, M ) ∩ {Σ = I}. For Σ = I, Q = β ⊺ Σβ is reduced to ∥β∥ 2 2 and (65) implies the following lower bound, It remains to establish the additional term of the lower bound M 2 / √ N + n, whose proof is based on the following version of Le Cam's Lemma (stated as Lemma 4 in Guo et al. [2017b]; See also LeCam [1973], Yu [1997], Ren et al. [2015]).
To establish the lower bound M 2 / √ N + n, we need to perturb the design covariance matrix and introduce the following null and alternative parameter spaces, where β ∈ R p satisfies ∥β∥ 0 ≤ k and ∥β∥ 2 = M and c = min Note that H 0 , H 1 ∈ Θ(k, M ). Since the conditional distribution f (y|X) is the same under both θ 0 and θ 1 , then we have the decompositions f θ 0 (y, X) = f (y|X)f θ 0 (X) and f θ 1 (y, X) = f (y|X)f θ 1 (X) and hence ∫ ∫ Hence, it is sufficient to control the L 1 or χ 2 distance between f θ 1 (X) and f θ 0 (X).
To control the distance, we introduce the following Lemma, which was established in Cai and Zhou [2012], Ren et al. [2015] and stated as Lemma 3 in Cai and Guo [2017b].

A.8. Proof of Corollary 6
The estimation bound (42) follows from the argument of Theorem 1 and the decomposition of (52). Note that the additional randomization term can be controlled as in (53). The proof of the coverage and precision properties (43) and (44) follows from the application of Corollary 4 to the following linear model, Note that the precision property also relies on the following observation, which follows from Lemma 9 and the definition of sub-exponential random variable.

B. Proof of Lemmas
To establish the technical lemmas, we introduce the following definitions. For a random variable U , its sub-gaussian norm is defined as ∥U ∥ ψ 2 = sup q≥1 and its sub-exponential norm is defined as ∥U ∥ ψ 1 = sup q≥1 1 q (E|U | q ) 1 q . For a random vector U ∈ R p , its sub-gaussian norm is defined as ∥U ∥ ψ 2 = sup v∈S p−1 ∥⟨v, U ⟩∥ ψ 2 and sub-exponential norm is defined as ∥U ∥ ψ1 = sup v∈S p−1 ∥⟨v, U ⟩∥ ψ1 , where S p−1 is the unit sphere in R p . The following lemma shows that the product of two sub-gaussian variables is a sub-exponential variable.
B.1. Proof of Lemma 1 The first inequality in (47) follows from the Holder's inequality while the second inequality holds under the event G 2 ∩ G 3 . On the event G 1 ∩ G 4 , the second error bound (48) follows from the following decomposition Together with (82), we establish (48). To establish (49), we start with the following decomposition, ) (84) In the following, we are going to bound the terms separately in the above decomposition. On the event G 1 , we have On the event G 6 ( β, β − β, t), we have On the event G 6 (β, By applying (85), (86) and (87) to the decomposition (84), we establish that with probability larger than 1 − p −c − γ(n) − e −ct 2 , .

B.2. Proof of Lemma 2
On the event G 5 (β, t)∩G 6 (β, β, t), the inequality (50) holds. The probability control of (50) follows from (83) with taking w = v = β. Let ρ n denote n/(N + n) and hence ρ n → ρ To establish (51), we start with the decomposition, By the above limiting distributions, together with the independence between the two terms on the right hand side of (88), we establish (51).

B.3. Proof of Lemma 3
The proof follows from that of Lemma 2. The main change is that In addition, we need to show that where Φ −1 denotes the inverse of quantile function for the standard normal random variable. By (82) and
We boundψ 2 − ψ 2 based on the following decomposition, Since EA 2 i = 1, it is sufficient to establish upper bounds for 1 . It follows from Lemma 9 that A 2 i a sub-exponential random variable. By Remark 5.18 in Vershynin [2012], A 2 i −EA 2 i is a sub-exponential random variable with sub-exponential norm smaller than 2M 1 ∥X i· ∥ 2 ψ 2 . By Corollary 5.17 in Vershynin [2012], we have ≤ 2 exp (−c log(n + N )) = 2(n+N ) −c .
(100) Since A i is a sub-gaussian random variable, there exists positive constants C 1 > 0 and c > 2 such that the following concentration inequality holds, ) . Then we have We control EÃ 4 i as follows, where the first inequality follows from Cauchy-Schwarz inequality, the second inequality follows from the fact that A i is a sub-gaussian random variable and the last inequality follows from (101). Now we apply Lemma 11 to bound 1 .

Proof of Equation
where the second term on the right hand side of (107) is further upper bounded by Note that (106) implies Then it is sufficient to control 1 , which is further decomposed as, Recall the definition of events in (80). On the event G 1 ∩ G 4 , ( β − β) ⊺ Σ S ( β − β) ≲ k log p/n; On the event G 1 ∩ G 4 ∩ G 6 (β, β, √ log p), It remains to control 4 ) 2 in the expression (110), which relies on the following fact. On the event G 1 , Ck log p ≤ 1 and hence ) 4 C 2 (k log p) 2 ≲ (k log p) 2 n + N .
Define the event B 1 as B is sub-gaussian random variable and β−β is independent of X i· for n+1 ≤ i ≤ n+N , then max i=1,2 P (B i ) ≲ (n + N ) −c .

B.6. Proof of Lemma 9
The proof for ∥U V ∥ ψ1 follows from the following inequality where the first inequality is by Cauchy-Schwarz and the second inequality follows from the definition of sub-gaussian norm. The proof of the centered part ∥U V − EU V ∥ ψ1 follows from the upper bound for ∥U V ∥ ψ1 and the remark 5.18 in Vershynin [2012].

C. Additional Simulation Results
In this section, we present the signal detection in the semi-supervised setting, as a supplement to the supervised results reported in Section 5.2. We consider the following generation of β as β j = δ for 1 ≤ j ≤ 50 and β j = 0 for 51 ≤ j ≤ 800 and vary δ across {0.00, 0.025, 0, 05, 0.075, 010, 0.125, 0.15} and vary the labelled sample size n across {600, 1200} and the unlabelled sample size N = 2000. In Figure 4, we demonstrate the coverage and precision properties of the randomized confidence intervals across four methods, the non-randomized detector D(0) and the three randomized detectors D(2), D(4) and D (6), where D(·) is defined in (35). The two plots on the top of Figure 4, corresponding to the semi-supervised setting with n = 600 demonstrate the effect of randomization on the empirical coverage and average lengths, where the randomization leads to a interval estimator achieving the coverage properties at the expense of wider interval estimators. With the randomization level τ 0 reaching 2, the coverage property is guaranteed while the empirical coverage for the procedure without randomization (τ 0 = 0) is much lower than 0.95. The bottom two plots of Figure 4 corresponds to the semi-supervised setting with n = 1, 200 and the main observation is similar to the case of n = 600 but the confidence intervals are much shorter than the setting with n = 600. In contrast to Figure 2, it shows that the additional unlabelled data helps reduce the average lengths of the confidence intervals.
The empirical detection rate in the semi-supervised setting is reported in Table  2, where the sample size n is generated across n = 600 and n = 1, 200 and the explained variance β ⊺ Σβ is controlled via the scaler δ. When δ = 0, it corresponds to the null case and a proper detection procedure is expected to have type I error rate 0.05. As predicted by theory, the detection method without randomization D(0) fails to give proper type I error due to presence of weak signals. With introducing the randomization procedure, the type I error rate gets closer to 0.05. When δ moves away from zero, the detection procedure is taken as a powerful procedure as the empirical detection rate approaches 1. For the detection procedure with randomization level τ 0 = 2, the setting with δ = 0.025 corresponds to an indistinguishable region, where it is challenging to detect the signal. However, as δ reaches 0.05, the detection rate reaches 0.800 for n = 600 and 0.944 for n = 1200. As characterized by theory, a larger randomization level requires a higher value of δ such that the signal can be detected, for example, for τ 0 = 4, only when δ reaches 0.075, the detection rate reaches 0.82 for n = 600 and 0.968 for n = 1200.