For instance, why is it that it is more favourable for a weight of [0.25, 0.25, 0.25, 0.25] (for which the L2 penalty is 0.25) instead of simply [1, 0, 0, 0] (for which an L2 penalty is 1)? In this case, both weights would give the same dot product when using W.T * X
Why does an L2 penalty prefer smaller and more diffuse weight vectors?
machine learning;svm;regularization
You answered this in your question. Prefer means produces a smaller penalty, and you've identified that the penalty in the first case is smaller. Why would this be a good thing? It amounts to preferring an explanation based a bit on many features, rather than one based entirely on one features. That's often a good bet to avoid overfitting.These two weight vectors do not produce the same dot product with other vectors in general. If the vector X contained all identical values, they would.If the 4 input features were regularly identical or nearly so, then it means they're redundant, and you may prefer to use just 1 of the features (your second case), instead of a bit of each. In this case, an L1 penalty would at least be indifferent to the two, not penalize the second one.