Why exactly does features being dependent on each other, features having high correlation with one another, mean that they would be redundant? Also, does PCA help get rid of redundant/irrelevant features or do we have to get rid of redundant/irrelevant features before running PCA on our dataset?
feature redundancy
feature selection;dimensionality reduction;pca
For the sake of training, features that are highly correlated offer little training value as the presence/state of one value can always (or almost always) be used to determine the presence/state of the other. If this is the case there's no reason to add both features as having both will have little impact on the predictions - if A on = B off, and A off = B on, then all states can be represented by just learning off either A or B. This is greatly simplified, but the same is true for other highly correlated values.PCA can help reduce features, but in any case, if you've identified redundant or highly correlated features that will be of little use in training, it probably makes sense to eliminate them right away and then use PCA, or other feature importance metrics that can be generated by training off your full dataset, to further optimize your training feature set.