<jupyter_start><jupyter_text>Law of the LAR's knotsIn this notebook, we present the law of the knots of the LAR. This result has been given in "*Multiple Testing and Variable Selection along Least Angle Regression's path*" (<NAME> <NAME>, 2019).  The recursive formulation of the LARWe begin with a new recursive algorithm, equivalent to LAR, that produces the sequence $(i_k,\lambda_k)$ of indexes $i_k$ and knots $\lambda_k$ of the LAR. This program has been implemnted under the name **lar_rec** given in the file *multiple_spacing_tests.py*.<jupyter_code>import numpy as np
import matplotlib.pyplot as plt
from multiple_spacing_tests import lar_rec, generate_variables<jupyter_output><empty_output><jupyter_text>The function **generate_variables** gives an istance of a high-dimensional regression problem with *predictors_number* the number of predictors, *sample_size* the sample size, *sparsity=0* the sparsity of the target vector (each nonzero entry is a Rademacher random variable $\{\pm 1\}$), $sigma=1$ the noise level. The matrix $Z$ is equal to $Z=X X^\top$ where $X$ is a design matrix given by drawing i.i.d. Gaussian rows. The matrix $R$ is given by $R=[Z, -Z; -Z, Z]$. The target vector is *beta_0* and the response variable is $Y$.Here, $p=250$ (predictors number) and $n=50$ (sample size).<jupyter_code>[Z,R,sigma,X,noise_correlation,sparsity,beta_0,Y]=generate_variables(predictors_number=250, sample_size=50)<jupyter_output><empty_output><jupyter_text>We compute the LAR using the recursive formulation. See also the notebook *multiple_spacing_tests.ipynb* for a comparison with SciKitLearn version of LAR.It returns the knots *lambas* and the indexes *indexes*.<jupyter_code>[lambdas, indexes, correls, R, Z] = lar_rec(X,Y)

k_max = np.size(indexes)

plt.plot(range(k_max-1),lambdas[:k_max-1])
plt.legend([r'recursive'])
plt.tight_layout()
plt.xlabel(r'$k$')
plt.ylabel(r'$\lambda_k$')
plt.show()<jupyter_output><empty_output><jupyter_text>The joint law of the knotsGiven the joint law of the knots proved in "*Multiple Testing and Variable Selection along Least Angle Regression's path*" (Azaïs and De Castro, 2019), one can compute the conditional law of $\lambda_b\|\lambda_a,\lambda_c$ whose cdf is denoted by $\mathbb F_{abc}$. In particuler, our theoretical analysis shows that $\hat\alpha_{abc}=\mathbb F_{abc}(\lambda_b)$ is uniformly distributed under the null (i.e., the target vector is zero). One can compute $\hat\alpha_{abc}$ using the function **observed_significance_CBC**. CBC means Component-By-Component as in "*Fast algorithms for component-by-component construction of rank-1 lattice rules in shift-invariant reproducing kernel Hilbert spaces*" by <NAME>, <NAME> iun Mathematics of Computation, 2006. It is a lattice rule to compute the conditional marginal distribution of $\lambda_b$. Here, we compute $\hat\alpha_{136}$.<jupyter_code>from multiple_spacing_tests import observed_significance_CBC

# compute the LAR
lars = [lambdas, indexes, correls]

# a (start), b (middle) and c (end)
start = 1
middle = 3
end = 6

# hat alpha 
alpha, __, __ = observed_significance_CBC(lars, sigma=sigma, start=start, end=end, middle=middle)

print("Observed significance is %s" % alpha)<jupyter_output>Observed significance is 0.5104805749849506
{'fraction_non_alphanumeric': 0.10433190970103721, 'fraction_numerical': 0.016168395363026236, 'mean_word_length': 6.051612903225807, 'pattern_counts': {'":': 0, '<': 20, '<?xml version=': 0, '>': 20, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '117487', 'license': 'MIT', 'n_tokens_mistral': 1097, 'n_tokens_neox': 991, 'n_words': 378, 'path': 'Law_LAR.ipynb', 'repo_name': 'ydecastro/lar_testing', 'size': '3289'}