Next, we study \( \max _{1\le k\le K} \big \Vert \widehat{B}^{(k)} \widehat{\theta }_\text{ols}^* -\widehat{\Sigma }_{xy}^{(k)*} \Vert  \) . Let \( \widehat{\Sigma }_{x\varepsilon } = N^{-1} \sum _{i=1}^{N} X_i\varepsilon _i\) , and \( \widehat{\Sigma }_{x\varepsilon }^{(k,m)} = n^{-1}\sum _{i\in \mathcal {S}^{(k, m)}} X_i \varepsilon _i \) .
Then, we should have \( \widehat{\Sigma }_{xy}^{(k,m)} = \widehat{\Sigma }_{xx}^{(k,m)} \theta _0 + \Sigma _{x\varepsilon }^{(k,m)} \) , and \(\widehat{\theta }_\text{ols} =\theta _0 + \widehat{\Sigma }_{xx}^{-1} \widehat{\Sigma }_{x\varepsilon }\) .
Note that \( \Vert  \widehat{\Sigma }_{xx}^{(k, m)}\widehat{\theta }_\text{ols} - \widehat{\Sigma }_{xy}^{(k, m)} \Vert  = \Vert  \widehat{\Sigma }_{xx}^{(k, m)} \widehat{\Sigma }_{xx}^{-1} \widehat{\Sigma }_{x\varepsilon } - \widehat{\Sigma }_{x\varepsilon }^{(k, m)} \Vert  \le \Vert  \widehat{\Sigma }_{xx}^{(k, m)} \Vert  \Vert  \widehat{\Sigma }_{xx}^{-1} \widehat{\Sigma }_{x\varepsilon } \Vert  + \Vert \widehat{\Sigma }_{x\varepsilon }^{(k, m)} \Vert \) .
Then by Bernstein’s inequality [1]}, we have \( P( \sqrt{N / \log K}\max _{1\le m\le M} \Vert \widehat{\Sigma }_{x\varepsilon }^{(k, m)} \Vert  \ge \nu )\le C_1 \exp \big [-C_2 \min \lbrace (\log K) \nu ^2, \sqrt{N\log K} \nu \rbrace \big ]\)  for any \( \nu > 0 \)  as long as \( p \)  and \( M \)  are fixed, where \( C_1, C_2 \)  are two positive constants. Hence, it follows that \( P(\sqrt{N / \log K} \max _{m,k} \Vert \widehat{\Sigma }_{x\varepsilon }^{(k, m)} \Vert  \ge \nu )\le C_1 \exp \big [\log K-C_2 \min \lbrace (\log K) \nu ^2, \sqrt{N\log K} \nu \rbrace \big ]\)  for any \( \nu > 0 \) . As a consequence, for any \( \delta >0 \) , there should be a sufficiently large constant \( \nu _\delta >0\)  such that \( P(\sqrt{N/\log K} \max _{m,k} \Vert \widehat{\Sigma }_{x\varepsilon }^{(k, m)} \Vert  \ge \nu _\delta )\le \delta \)  for each \( N \) , as long as \( \log K =o(N)\) . Thus, we should have \( \max _{m,k} \Vert \widehat{\Sigma }_{x\varepsilon }^{(k, m)} \Vert  = O_p(\sqrt{(\log K) / N}) \) .
Since \( \max _{k,m} \Vert \widehat{\Sigma }_{xx}^{(k, m)} \Vert =O_p(1) \)  and \( \Vert  \widehat{\Sigma }_{xx}^{-1} \widehat{\Sigma }_{x\varepsilon } \Vert  = O_p( 1/\sqrt{N}) \) , we can conclude that \( \max _{1\le k\le K} \big \Vert \widehat{B}^{(k)} \widehat{\theta }_\text{ols}^* -\widehat{\Sigma }_{xy}^{(k)*} \Vert  =O_p(\sqrt{(\log K) / N})\)  as long as \( \log K = o(N) \) .
