<jupyter_start><jupyter_text>DATA ANALYSIS WITH PANDAS<jupyter_code>#to find hardest day to work? (based on number of tables been served)
df.groupby('day').count()<jupyter_output><empty_output><jupyter_text>Lets find out what is the best day to work - maximum tips (sum and  percents)<jupyter_code>df2 = df.groupby('day').sum() #sum per day (jo jo numerial wale h unka sum milega)
#print(df2)
df2.drop('size',inplace=True,axis=1) #sum of size column is not relevant(inplace by default is false.. kuch chng krna h to true krdia.. and ek hi axis pe data hai)
df2['percent'] = df2['tip']/df2['total_bill']*100 #add percents. adding percent column in df2

df2.to_csv('df2.csv') #to store in csv file
df2
#if upr false likhdia to us case me sirf show ni krega next run me. further run me krega show.. that is wo temporary chng krra h
#axis 0 tab dete h jb row delete krni ho
df3 = df.groupby('smoker').sum() #smoker ke hisaab se sum krdia
df3['percent'] = df3['tip']/df3['total_bill']*100
df3.to_csv('df3.csv')
df3<jupyter_output><empty_output><jupyter_text>Lets groupby day and size<jupyter_code>df4 = df.groupby(['day','size']).sum()
df4['percent'] = df4['tip']/df4['total_bill']*100
df4.to_csv('df4.csv')
df4.dropna() #drop null rows taaaki calculation shi ho<jupyter_output><empty_output><jupyter_text>Visualization with seaborn<jupyter_code>#lets draw some seaborn graphs:

#Tables per day

sb.countplot(x='day',data=df)
py.show()
#tables per day per size
sb.countplot(x='day',hue='size',data=df) #hue is to divide the x into parts
py.show()<jupyter_output><empty_output><jupyter_text>Transform and clean the dataBefore we start building our model, we need to convert all the text values into numbers. We can do it in many ways.Using update statements Using replace method iterate over the rows Use dummy variables Using replace Convert sex and smoker columns to values<jupyter_code>df.replace({'sex':{'Male':0 , 'Female':1},'smoker':{'No':0 , 'Yes':1}}, inplace=True)
df.to_csv('df_replaced.csv')
df.head()<jupyter_output><empty_output><jupyter_text>Using dummy variables: The values in a day column are: Thu, Fri, Sat, Sun we can convert it to 1,2,3,4 but to get a good model, it is better to use boolean variables. We can achieve it by converting the column into 4 column - one for each day with 0 or 1 as values. In pandas library it can be done using get_dummies():<jupyter_code>days = pd.get_dummies(df['day']) #convert categorical values into dummy/indicator variables #days ko column me krdia.. if bht saari values hoti ek column me to it would have been difficult to change all to some integer values
days.sample(5)<jupyter_output><empty_output><jupyter_text>Actually we can drop columns without loosing the data.. eg thursday ni chahiye<jupyter_code>days = pd.get_dummies(df['day'],drop_first = True) #pehli column ko drop krdia  (feature minimization.. kyuki bina thusday ke bhi pta chl jayga ki thursday h ya ni.. jab sb 0 ho to thursday hoga)
days.sample(6)<jupyter_output><empty_output><jupyter_text>Do the same with time column and concat all data frames. Also we dont need the day and time columns anymore so we drop them<jupyter_code>days = pd.get_dummies(df['day'],drop_first=True)
df = pd.concat([df,days],axis = 1) #df me days add krdie(dummies wale)
times = pd.get_dummies(df['time'],drop_first=True)
df = pd.concat([df,times],axis=1) #df me times add krdie(dummies wale)
df.drop(['day','time'],inplace=True,axis=1) #days and time ko drop krdia original data me se
df.to_csv('df_drop.csv')
df.head()<jupyter_output><empty_output><jupyter_text>Building Our Machine Learning Model<jupyter_code>X=df[['sex','smoker','size','Fri','Sat','Sun','Dinner']] #input/features
Y=df[['tip']] #output<jupyter_output><empty_output><jupyter_text>now lets split the data into test and train so we can test our model before we use it - we decide to split 70% - 30%<jupyter_code>from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
X_train, X_test, y_train, y_test = train_test_split(X,Y,test_size=0.25,random_state=26) #isme 25 % le lia #random_state seed ki trh work krra h ki ek br data split kardia randomly to whi rhega.. seed ki trh value h kuch uski
model = LinearRegression()
model.fit(X_train, y_train)
print(len(X_test))
predictions = model.predict(X_test) #X_test dia usse prediction aygi
print(predictions)
#we can now look at the predictions and compare it with y_test
#we can draw a graph to see the difference distribution:
sb.distplot(y_test-predictions)#it is difference distribution #asli values h hmaare paas y_test me.. un dono me plot krdia graph
py.show()<jupyter_output><empty_output><jupyter_text>We can see from the graph that most of the times the predictions were correct (difference = 0). We can continue working on the model , adding data and play with the parametersIf we want to predict new value for example :We have a 3 size table smoker male on friday lunch:<jupyter_code>#predict using own values 
#if we want to predict new value for example:
#we have a 3 size table smoker male on friday lunch
myvals = np.array([0,1,3,1,0,0,0]).reshape(1,-1) #to convert into multidimentional array we use reshape(1,-1).. #array me individual value di h hr column ki of X...
print(myvals)
myvals1 = np.array([0,1,3,1,0,0,0]) #if ye dedete to aisa kuch hota.. mgr upr waala dete h
print(myvals1)
 model.predict(myvals) #prediction
#array me aari h kyuki ek hi data h
#and the same table on dinner:
myvals = np.array([0,1,3,1,0,0,1]).reshape(1,-1) #([sex=Male,smoker=yes,size=3,friday,not sat,not sun,time=dinner])
a=model.predict(myvals)
print(a)<jupyter_output>[[ 3.73414562]]
{'fraction_non_alphanumeric': 0.11117011860506285, 'fraction_numerical': 0.01858736059479554, 'mean_word_length': 5.932515337423313, 'pattern_counts': {'":': 0, '<': 44, '<?xml version=': 0, '>': 44, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '519843', 'license': 'MIT', 'n_tokens_mistral': 2011, 'n_tokens_neox': 1943, 'n_words': 682, 'path': 'day13/linear_regression_python_machine_learning.ipynb', 'repo_name': 'AkshayManchanda/Machine_Learning_with_Python_Training', 'size': '5649'}