(Note that the error is the error relative to the target , not the error relative to the testing **data** . The difference between these two errors is random noise, known as irreducible prediction error.) Typically there is a trade-off between bias and variance -decrease in one usually causes increase in the other -but we have seen here that it is sometimes possible to minimize both simultaneously.
1 4 ⁄ d i t i ≠ d i t i = squared error E g i t i - ( ) 2 ( ) = E g i t i - ( ) 2 ( ) E g i ( ) t i - ( ) 2 E E g i ( ) g i - ( ) 2 ( ) + = E g i t i - ( ) 2 ( ) t i E g i β i - ( ) 2 ( ) β i