<jupyter_start><jupyter_text>Feature Selection is the technique which is used to distinguish the best feature for prediction.It is useful for model training speed and loss reduction in extracting those features more relevant for prediction of our target ------ Feature Selection is done for the sake of:1. Reduces overfitting or avoid underfitting while training performed2. Improves Accuracy(clasification) or decrease Loss(regression)3. It improves the time complexity<jupyter_code>url='https://raw.githubusercontent.com/jbrownlee/Datasets/master/pima-indians-diabetes.csv'<jupyter_output><empty_output><jupyter_text><jupyter_code>import pandas as pd
import numpy as np
data=pd.read_csv(url)
from numpy import set_printoptions

from sklearn.feature_selection import SelectKBest
from sklearn.feature_selection import f_classif
names=['preg','plas','pres','skin','test','mass','pedi','age','class']
data1=pd.read_csv(url,names=names)<jupyter_output><empty_output><jupyter_text>**Load Our Dataset**<jupyter_code>from numpy import set_printoptions
from sklearn.feature_selection import SelectKBest
from sklearn.feature_selection import f_classif
names=['f1','f2','f3','f4','f5','f6','f7','f8','f9']
dats=pd.read_csv(url,names=names)
dats.columns
dats.head(10)
array=dats.values
x=array[:,0:8]
y=array[:,8]<jupyter_output><empty_output><jupyter_text>**SELECT K_BEST** This is Uni variate feature selection<jupyter_code>test=SelectKBest(score_func=f_classif,k=4)
fit=test.fit(x,y)
set_printoptions(precision=3)
print(fit.scores_)
features=fit.transform(x)
features.shape

for i in range(4):
  print(features[i])<jupyter_output>[  6.  148.   33.6  50. ]
[ 1.  85.  26.6 31. ]
[  8.  183.   23.3  32. ]
[ 1.  89.  28.1 21. ]<jupyter_text>Therefore We can see that the features are reduced  **RECURSIVE FEEATURE ELIMINATION METHOD** RFE is is wrapper used to train model and give ranks for each feature then choose the specified number of features.The training is done on all or group of features to give the importance score for selection.<jupyter_code>from sklearn.feature_selection import RFE
from sklearn.linear_model import LogisticRegression
names = ['preg', 'plas', 'pres', 'skin', 'test', 'mass', 'pedi', 'age', 'class']
d1=pd.read_csv(url,names=names)
array=d1.values
x=array[:,0:8]
y=array[:,8]
model=LogisticRegression(solver='lbfgs')
rfe=RFE(model,3)
fit=rfe.fit(x,y)
print("Num features:%d"%fit.n_features_)
print("Selected Features: %s"%fit.support_)
print("Feature Ranking:%s"%fit.ranking_)<jupyter_output>Num features:3
Selected Features: [ True False False False False  True  True False]
Feature Ranking:[1 2 4 5 6 1 1 3]<jupyter_text>We can see That the first faeture and the second and the third  features from the last are selected. **PRINCIPAL COMPONENT ANALYSIS** PCA like its name tells it finds components of planes containing vectors which contains much of our data's distribution.<jupyter_code>from sklearn.decomposition import PCA
array=d1.values
x=array[:,0:8]
y=array[:,8]
pca=PCA(n_components=3)
fit=pca.fit(x)
print("explained variance: %s"%fit.explained_variance_ratio_)
print(fit.components_)
#PCA with the ratio of variance we want to preserve from our data
pca_percentage=PCA(n_components=0.90)
fit=pca.fit(x)
print("explainde variance: %s"%fit.explained_variance_ratio_)
print(fit.components_)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1178400954653938, 'fraction_numerical': 0.023269689737470168, 'mean_word_length': 7.158150851581508, 'pattern_counts': {'":': 0, '<': 23, '<?xml version=': 0, '>': 23, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '273980', 'license': 'MIT', 'n_tokens_mistral': 1176, 'n_tokens_neox': 1136, 'n_words': 308, 'path': 'FEATURE_SELECTION.ipynb', 'repo_name': 'mintesin/Portofolio-Projects', 'size': '3352'}