I have a training set with columns as follows:$x_1,x_2,x_3,x_4, x_3/x_4$. I want to know if I should consider $x_3$,$x_4$ and $x_3/x_4$ as separate features (as $x_3/x_4$ is derived from other features)? Also I found that dropping $x_3$ as a feature increases my k-fold neg-loss score (marginally!).The underlying question is how would I know if entertaining $1/x_1$ or $x_1/x_2$ or even $\sqrt{x_4}x_3^4/x_1$... (you get the idea) as another feature column will/won't increase my score?Is there any method/algorithm to know which modified features will increase my score?
Regarding modification of feature columns in supervised learning
machine learning;feature selection;supervised learning;feature engineering
I agree mostly with what was already said regarding feature engineering and just to provide you with more material this post has a nice analysis regarding different stages in feature engineering, with many links and references to papers and specific challenges. I think it's worth checking out.Also, to some extend you could automate the task of finding good compound features, using kernel methods. You could use some of the standard kernel methods implemented in many libraries (e.g. in sklearn) as feature extractors and feed these higher-level features as input to a random forest model that inherently uses feature-importance while training. Then keep only most-informative of these higher-dimension features, as scored by the trained model, as compound features alongside the raw data.  