<jupyter_start><jupyter_text>Author : <NAME> Topic : All about Feature Engineering  Email : <EMAIL>  What is Feature Selection TechniquesFeature Selection is one of the core concepts in machine learning which hugely impacts the performance of your model. The data features that you use to train your machine learning models have a huge influence on the performance you can achieve.Irrelevant or partially relevant features can negatively impact model performance.  Benefits of performing feature selection before modeling your data?Reduces Overfitting: Less redundant data means less opportunity to make decisions based on noise.Improves Accuracy: Less misleading data means modeling accuracy improves.Reduces Training Time: fewer data points reduce algorithm complexity and algorithms train faster.  Feature Selection Methods:1. Univariate Selection2. Feature Importance3. Correlation Matrix with Heatmap  1. Univariate SelectionSome Function:    1. chi-square : Best selection for categorial data    2. f_classif    3. mutual_info_classif  Import Necessary Libraries<jupyter_code>import numpy as np
import pandas as pd
import matplotlib.pyplot as plt<jupyter_output><empty_output><jupyter_text>Import Dataset<jupyter_code>df = pd.read_csv(r'F:\Feature Engineering\Univariate Selection\cardio_train.csv',sep = ";")
df
df.shape
df['yr'] = (df['age']/365).round(0)
df['yr']
df.describe()
df.corr()
df = df.drop(['yr'],axis = 1)
x = df.iloc[:,:-1]
y = df.iloc[:,12]
y
from sklearn.feature_selection import SelectKBest
from sklearn.feature_selection import f_classif
FIT_FEATURES = SelectKBest(score_func = f_classif)
FIT_FEATURES.fit(x,y)
pd.DataFrame(FIT_FEATURES.scores_)
SCORE_COL = pd.DataFrame(FIT_FEATURES.scores_,columns = ['score value'])
SCORE_COL
NAME_COL = pd.DataFrame(x.columns)
pd.DataFrame(NAME_COL)
topr_features = pd.concat([NAME_COL,SCORE_COL], axis =1)
topr_features
topr_features.nlargest(8,'score value')
from sklearn.model_selection import train_test_split
xtrain,xtest,ytrain,ytest = train_test_split(x,y,test_size=.3,random_state = 1)
xtrain
from sklearn.ensemble import RandomForestClassifier
rfc = RandomForestClassifier()
rfc.fit(xtrain,ytrain)
rfc.score(xtest,ytest)<jupyter_output><empty_output><jupyter_text>Feature Importance<jupyter_code>from sklearn.ensemble import ExtraTreesClassifier
model = ExtraTreesClassifier()
model.fit(x,y)
model.feature_importances_
top = pd.Series(model.feature_importances_,index = x.columns)
top
top.nlargest(10).plot(kind = 'barh')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08719778042013476, 'fraction_numerical': 0.008323424494649227, 'mean_word_length': 6.987341772151899, 'pattern_counts': {'":': 0, '<': 15, '<?xml version=': 0, '>': 15, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '160902', 'license': 'MIT', 'n_tokens_mistral': 807, 'n_tokens_neox': 783, 'n_words': 222, 'path': 'Feature Engineering/Feature Selection Technique/Feature Selection Technique.ipynb', 'repo_name': 'jrderek/Data-science-master-resources', 'size': '2545'}