<jupyter_start><jupyter_text>__References___ https://seaborn.pydata.org/tutorial/regression.html Using Flask to serve a machine learning model as a RESTful webservice https://www.youtube.com/watch?v=s-i6nzXQF3g  [Accessed: December 15, 2020]Deploy Machine Learning model Flask on web page https://www.youtube.com/watch?v=i3RMlrx4ol4 [Accessed: December 17, 2020]<jupyter_code>import pandas as pd
import sklearn as sklearn
import seaborn as sns
import numpy as numpy
import sklearn.linear_model as lin
#stackoverflow chat suggests that you can't import an external dataset into seaborn, use pandas instead??!!
#It's as though if it's not here >>>https://github.com/mwaskom/seaborn<<< seaborn doesn't want to know.
powerproduction=pd.read_csv("powerproduction.csv")
#powerproduction.describe()
#print(powerproduction)<jupyter_output><empty_output><jupyter_text>Analysis 1<jupyter_code>#sns.pairplot(powerproduction)<jupyter_output><empty_output><jupyter_text>Functions to draw linear regression models<jupyter_code>sns.regplot(x="speed", y="power", data=powerproduction);
#sns.regplot(x="power", y="speed", data=powerproduction);
#a blue line appears as a suggestion<jupyter_output><empty_output><jupyter_text>Source: https://seaborn.pydata.org/tutorial/regression.htmlYou should note that the resulting plots are identical, except that the figure shapes are different.regplot() accepts the x and y variables in a variety of formats including simple numpy arrays, pandas Series objects, or as references to variables in a pandas DataFrame object passed to data. In contrast, lmplot() has data as a required parameter and the x and y variables must be specified as strings. This data format is called “long-form” or “tidy” data. Other than this input flexibility, regplot() possesses a subset of lmplot()’s features, so we will demonstrate them using the latter.Observations on these graphsThe curve shape corresponds with graphs at https://energyeducation.ca/encyclopedia/Wind_power It is logical, it takes a reasonable amount of wind to get the turbine going and then when it's moving and the wind drops...it will still rotate and come to a stop eventually.The cut off point seems to be around speed 24.5 kmph.Turbines are designed to operate within a specific range of wind speeds. The limits of the range are known as the cut-in speed and cut-out speed.The cut-in speed is the point at which the wind turbine is able to generate power. The cut-out speed is the point at which the turbine must be shut down to avoid damage to the equipment. The cut-in and cut-out speeds are related to the turbine design and size and are decided on prior to construction.  TrainAttempt to automate the prediction, find relationships between the paired data.<jupyter_code>import sklearn.linear_model as lin #manipulate the two lists of numbers x = flipper["body_mass_g"].to_numpy() y = flipper["flipper_length_mm"].to_numpy() #even though you've only one input value, you must reshape as if there are more. It is ascikit learm thing.... x = x.reshape(-1, 1) #use sckitlearn to give numbers pertaining to the suggested blue line above model = lin.LinearRegression() model.fit(x, y)#tells skcitlearn where the values are r = model.score(x, y)#find out the r value, how well the lines fits the data set p = [model.intercept_, model.coef_[0]]#provide the intercept

powerproduction=pd.read_csv("powerproduction.csv")
#manipulate the two lists of numbers

#x=powerproduction["speed"].to_numpy()
speed=powerproduction["speed"].to_numpy()
y=powerproduction["power"].to_numpy()
#y = data["power"].to_numpy()
#X (windspeed) is the independent variable and y (power generated) is the dependent one
x = speed.reshape(-1, 1)
model = lin.LinearRegression()
model.fit(x, y)#tells skcitlearn where the values are
r = model.score(x, y)#find out the r value, how well the lines fits the data set
p = [model.intercept_, model.coef_[0]]#provide the intercept
r
#In statistics, the correlation coefficient r measures the strength 
#and direction of a linear relationship between two variables on a scatterplot.
#.72 is not too bad of a fit
p
#The Importance of Intercept
#https://www.analyticsinsight.net/why-is-intercept-important-in-regression-analysis/
#The intercept (often labeled as constant) is the point where the function crosses the y-axis. 
#In some analysis, the regression model only becomes significant 
#when we remove the intercept, and the regression line reduces to Y = bX + error.<jupyter_output><empty_output><jupyter_text>Predict<jupyter_code>def f(x, p):
# x is the input, p is the parameter/s e.g. a list of values somehow trained on a dataset already
# p can be used to help us make predictions in the case of x
 return p[0] + x * p[1]
#this function is designed to provide linear results
#that is correct because the dataset is linear
#per Ian..... the calculations are straightforward,our ideas behind them are important. 
#the functions might be deterministic maybe the input becomes part of the function (not external data anymore)
f(7, p)
#we can use the p values above where 7 windspeed, how much power is generated?
#we trained p on the dataset, it is a model

#you could define another function using p also , see below.
def predict(x):
    return f(x, p)

predict(2.9)
#0.3611247991539184<jupyter_output><empty_output><jupyter_text>The web service needs to get input into the function. The function needs to reject amounts higher than 24.399 The function needs to reject amounts lower than 0.325 Breakthrough!When I leave the dataset as it is , the figures are skewed.Remove the problematic values to train it properly<jupyter_code>powerproduction=pd.read_csv("powerproduction.csv")
#Code adapted from https://stackoverflow.com/questions/22649693/drop-rows-with-all-zeros-in-pandas-data-frame
df=powerproduction
#print (df.sort_values('power', ascending=True))
new_df = df[df.loc[:]!=0].dropna()
#df.drop(0,111,110,105,89) #this does not work - only deletes index 0
#print(df.drop(['speed'], axis=1))
new_df
#451 rows × 2 columns
#new_df.to_csv(index=False)
newdf_csv_data = new_df.to_csv('new_df.csv', index = False) 
new_df.describe()<jupyter_output><empty_output><jupyter_text>Analysis 2Start again with this better dataset<jupyter_code>sns.pairplot(new_df)
sns.lmplot(x="speed", y="power", data=new_df);

import sklearn.linear_model as lin
powerproduction=pd.read_csv("new_df.csv")
#manipulate the two lists of numbers
#x=powerproduction["speed"].to_numpy()
#X (windspeed) is the independent variable and y (power generated) is the dependent v
speed=powerproduction["speed"].to_numpy()
y=powerproduction["power"].to_numpy()
#y = data["power"].to_numpy()

x = speed.reshape(-1, 1)
model = lin.LinearRegression()
model.fit(x, y)#tells skcitlearn where the values are
r = model.score(x, y)#find out the r value, how well the lines fits the data set
p = [model.intercept_, model.coef_[0]]#provide the intercept
r
p
def f(x, p):
# x is the input, p is the parameter/s e.g. a list of values somehow trained on a dataset already
# p can be used to help us make predictions in the case of x
    return p[0] + x * p[1]
#this function is designed to provide linear results
f(7, p)
def predict(x):
    return f(x, p)
predict(2.9)<jupyter_output><empty_output><jupyter_text>A thought What if the relationship isn't LINEAR, the plot is certainly trying to tell me this.Also - I'm not doing groundbreaking research here surely someone has done this before??!!And has figured out the regression. A Clifton et al in 2013 carried out indepth research to conclude that :The power output from the turbine is not a linear function of wind speed, it is a complex non-linear function of hub height wind speed, turbulence intensity, and rotor disk shear.They instead proposed a machine learning technique called 'regression trees'that is implemented in MATLABThe regression tree method predicts wind turbine energy capture with two to three times more accuracy than the industry-standard power curve method. Not so goodI tried various sample codes to do a regression tree method and they are not working.They all need two inputs to predict - it appears to work best at classification rather than prediction.<jupyter_code>#Adapted from https://www.codespeedy.com/decision-tree-regression-in-python-using-scikit-learn/
import pandas as pd
import sklearn
from sklearn import tree
from sklearn.tree import DecisionTreeRegressor
cleanpowerproduction_df = pd.read_csv("new_df.csv")
cleanpowerproduction_df.columns
#Now we call our target value for which prediction is to be made.
#hgt = data.Height.values.reshape(-1,1)
#wgt = data.Weight.values.reshape(-1,1)
y = cleanpowerproduction_df.power.values.reshape(-1,1)

#The columns that out inputted into our model are known as 'features.
#These columns are used to determine the home price.

#For now, we will build our model for only a few features.

cleanpowerproduction_features = ['speed', 'power']

X = cleanpowerproduction_df[cleanpowerproduction_features]

# Defining. model.
cleanpowerproduction_model = DecisionTreeRegressor(random_state=42)

# Fit the model
cleanpowerproduction_model.fit(X, y)
#r = cleanpowerproduction_model.score(X, y)#find out the r value, how well the lines fits the data set
#p = [model.intercept_, model.coef_[0]]#provide the intercept
#We make predictions of Price for first 5 houses using Decision Tree regressor

print("The predictions for following following 5 houses:")
print(X.tail())
print("The predictions made for houses are : ")
print(cleanpowerproduction_model.predict(X.tail()))
#print(cleanpowerproduction_model.predict(X.all()))
#cleanpowerproduction_model.predict(X.tail())
#cleanpowerproduction_model.predict([[17,1]])
#cleanpowerproduction_model.predict([[17]])

#Adapted from https://www.geeksforgeeks.org/python-decision-tree-regression-using-sklearn/
# import numpy package for arrays and stuff 
import numpy as np  
# import matplotlib.pyplot for plotting our result 
import matplotlib.pyplot as plt 
# import pandas for importing csv files  
import pandas as pd 
#Step 2: Initialize and print the Dataset.
# import dataset 
dataset = pd.read_csv("new_df.csv") 
# print the dataset 
print(dataset) 
#Step 3: Select all the rows and column 1 from dataset to “X”
# select all rows by : and column 1 
# by 1:2 representing features 
#X = dataset[:, 1:2].astype(int) 
X = dataset.power.astype(int) 

#X = dataset[dataset_features] 
# print X 
print(X)
#Step 4: Select all of the rows and column 2 from dataset to “y"
# select all rows by : and column 2 
# by 2 to Y representing labels 
#y = dataset[:, 1].astype(int)  
y = dataset.speed
#y.reshape(-1, 1)
# print y 
#print(y)
# create a regressor object 
#regressor = DecisionTreeRegressor(random_state = 0)  
  
# fit the regressor with X and Y data 
#regressor.fit(X, y) 

# predicting a new value 

# test the output by changing values, like 3750 
#y_pred = regressor.predict([[8]]) 

# print the predicted price 
#print("Predicted price: % d\n"% y_pred)<jupyter_output><empty_output><jupyter_text>Maybe polynomial is a better fit What is Regression?Using the relationship between variables to find the best fit line or the regression equation that can be used to make predictions”. (https://towardsdatascience.com/introduction-to-linear-regression-and-polynomial-regression-f8adc96f31cb)Polynomial RegressionPolynomial Regression is a form of linear regression in which the relationship between the independent variable x and dependent variable y is not linear but it is the nth degree of polynomial.adapted from https://iq.opengenus.org/polynomial-regression-using-scikit-learn/In this example we will apply linear regression as well as polynomial regression on the same dataset and will see the results of both the models.<jupyter_code>#Adapted from https://iq.opengenus.org/polynomial-regression-using-scikit-learn/
#importing the libraries
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline
#importing the dataset
polydataset=pd.read_csv('new_df.csv')
#polydataset.info
#Now we divide the dataset into X and y,where X is the independent variable (windspeed) and y (power produced) is the dependent variable .
#X=dataset.iloc[:,1:2].values
#y=dataset.iloc[:,2].values
#X = polydataset.speed.reshape(-1, 1)
#This numpy array and reshaping stuff is to avoid errors later on.
speed = polydataset.speed.to_numpy()
y = polydataset.power.to_numpy()


#speed=powerproduction["speed"].to_numpy()
#y=powerproduction["power"].to_numpy()
#y = data["power"].to_numpy()
X = speed.reshape(-1, 1)
#model = lin.LinearRegression()
#fitting the linear regression model to the dataset
from sklearn.linear_model import LinearRegression
lin_reg=LinearRegression()
lin_reg.fit(X,y)

r = lin_reg.score(X,y)#find out the r value, how well the lines fits the data set
#p = [lin_reg.intercept_, lin_reg2.coef_[0]]#provide the intercept

#Adapted from Machine Learning Tutorial Python - 7: Training and Testing Data
#https://www.youtube.com/watch?v=fwY9Qv96DJY
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.3)
len(y_test)
from sklearn.linear_model import LinearRegression
classifier=LinearRegression()
classifier.fit(X_train, y_train)
classifier.predict(X_test)
classifier.score(X_test,y_test)
r
p
Training data and test data are two important concepts in machine learning.
The training data consists of observations (one observed input and one output variable in this case).
the algorithm uses training data to learn. 
The test set is a different set of observations used to evaluate the performance of the model.
Ensure that the test set does contain examples from the training set, it will be difficult to assess whether the algorithm is working correctly.
It is common to partition a single set of supervised observations into training and test sets.
Polynomial Feature Transform
Polynomial features are those features created by raising existing features to an exponent.
A squared or cubed version of an input variable will change the probability distribution.This separation can help some machine learning algorithms make better predictions and is common for regression predictive modeling tasks and generally tasks that have numerical input variables.

Typically linear algorithms, such as linear regression and logistic regression, respond well to the use of polynomial input variables.
# perform a polynomial features transform of the dataset
trans = PolynomialFeatures(degree=3)
data = trans.fit_transform(data)
# convert the array back to a dataframe
dataset = DataFrame(data)
# summarize
print(dataset.shape)
Running the example performs the polynomial features transform on the sonar dataset.

We can see that our features increased from 61 (60 input features) for the raw dataset to 39,711 features (39,710 input features).

(208, 39711)
1
(208, 39711)
#### test and train today
#Adapted from Machine Learning Tutorial Python - 7: Training and Testing Data
from sklearn.model_selection import train_test_split


# Split the data set observations into 70 % train 30% test
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.3)
#len(X_train)

##fit the polynomial regression model to the training dataset

from sklearn.preprocessing import PolynomialFeatures
from sklearn import model_selection
poly_reg=PolynomialFeatures(degree=4)
X_trainpoly=poly_reg.fit_transform(X_train)
polyreg=LinearRegression()
polyreg.fit(X_trainpoly, y_train)

X_test_poly = poly_reg.fit_transform(X_test)

y_predict_poly = polyreg.predict(X_test_poly)

poly_reg.score(X_test, y_test)
#fitting the polynomial regression model to the dataset
from sklearn.preprocessing import PolynomialFeatures
poly_reg=PolynomialFeatures(degree=4)
X_poly=poly_reg.fit_transform(X)
poly_reg.fit(X_poly,y)
polyreg=LinearRegression()
polyreg.fit(X_poly,y)


r = polyreg.score(X_poly,y)#find out the r value, how well the lines fits the data set
p = [polyreg.intercept_, polyreg.coef_[0]]#provide the intercept
r
#the best possible score is 1.0
polyp = [polyreg.intercept_, polyreg.coef_[0]]#provide the intercept
polyp
#Visualising the linear regression model results
plt.subplot(1,2,1)
plt.scatter(X,y,color='seagreen')
plt.plot(X,lin_reg.predict(X),color='darkgrey')
plt.title('Linear Regression')
plt.xlabel('Power')
plt.ylabel('Speed')

#Visualising the polynomial regression model results
plt.subplot(1,2,2)#2 is passed in so that this subplot will appear as the second one
X_grid=np.arange(min(X),max(X),0.1)
X_grid=X_grid.reshape((len(X_grid),1))
plt.scatter(X,y,color='seagreen')
plt.plot(X,polyreg.predict(poly_reg.fit_transform(X)),color='darkgrey')
plt.title('Polynomial Regression')
plt.xlabel('Power')
plt.ylabel('Speed')
plt.tight_layout()
plt.subplots_adjust(top=0.88)
plt.suptitle('Power Production')


#Visualising the pollynomial regression model results
#plt.subplot(1,2,2)
#X_grid=np.arange(min(X),max(X),0.1)
#X_grid=X_grid.reshape((len(X_grid),1))
#plt.scatter(X,y,color='green')
#plt.plot(X,lin_reg2.predict(poly_reg.fit_transform(X)),color='blue')
#plt.title('Power Production(Polynomial Regression)')
#plt.xlabel('Power')
#plt.ylabel('Speed')
#plt.show()
#oh yes that is a sweet looking fit!
#predicting the result of linear regression model.
lin_reg.predict( ([ [20.5] ]) )
#predicting the result of polynomial regression.
polyreg.predict( poly_reg.fit_transform(([ [20.5] ]) ))<jupyter_output><empty_output><jupyter_text>Turn the code into a function As above and in the "Functions and models" video tutorial, the co-efficent value can be fed into functionMake sure the right p value is passed in.  predictlinear()Is it possible to use the P value that I have figured out and simply do a calculation to return a linear prediction?$f(x,p)$<jupyter_code>def f(x, p):
# x is the input, p is the parameter/s e.g. a list of values somehow trained on a dataset already
# p can be used to help us make predictions in the case of x
    return p[0] + x * p[1]
#this function is designed to provide linear results

f(6.5,p)
def predict(x):
    return f(x, p)
#this version probably easier to use in flask app file...?
predict(6.5)
#https://github.com/ianmcloughlin/jupyter-teaching-notebooks/blob/master/keras-linear.ipynb
# Let's set f to a polynomial instead.
#f = lambda x: 2.0 * x**2 + 3.0 * x + 4.0
def predictpoly(g):
    import cmath
    p = [polyreg.intercept_, polyreg.coef_[0]]#provide the intercept
    return (2.0 * p**2) + (3.0 * p + 4.0)
predict()
def f(x, p):
# x is the input, p is the parameter/s e.g. a list of values somehow trained on a dataset already
# p can be used to help us make predictions in the case of x
    return p[0] + x * p[1]
#this function is designed to provide linear results

f(8.5,p)<jupyter_output><empty_output><jupyter_text>get the web service to use the function and dataset directly  Saving the model: Serialization and Deserialization using sklearn’s joblibhttps://www.datacamp.com/community/tutorials/machine-learning-models-api-pythonjoblib.dump(value, filename, compress=0, protocol=None, cache_size=None)Persist an arbitrary Python object into one file.Returns:	filenames: list of stringsThe list of file names in which the data is stored. If compress is false, each array is stored in a different file.<jupyter_code>import joblib
#from sklearn.externals import joblib
#joblib.dump(lr, 'model.pkl')
joblib.dump(polyreg, 'model.pkl')
#print("Model dumped!")
The Logistic Regression model is now "persisted". 
Perist in this context means having it available for future use without having to retrain.
You can load this model into memory with a single line of code. 
Loading the model back into your workspace is known as Deserialization.
lr = joblib.load('model.pkl')
#You’re now ready to use Flask to serve your persisted model.<jupyter_output><empty_output><jupyter_text>Creating an API from a machine learning model using FlaskFor serving your model with Flask, you will do the following two things:1. Load the already persisted model into memory when the application starts,2. Create an API endpoint that takes input variables, transforms them into the appropriate format, and returns predictions.<jupyter_code>Let's write a function predict() which will do:

Load the persisted model into memory when the application starts,
Create an API endpoint that takes input variables, transforms them into the appropriate format, and returns predictions.
You have already seen how to load a persisted model. Now, you will focus on how you can use it for predicting upon receiving inputs.
from flask import Flask, jsonify
app = Flask(__name__)
@app.route('/predict', methods=['POST'])
def predict():
     json_ = request.json
     query_df = pd.DataFrame(json_)
     query = pd.get_dummies(query_df)
     prediction = lr.predict(query)
     return jsonify({'prediction': list(prediction)})
#https://developer.ibm.com/technologies/artificial-intelligence/tutorials/deploy-a-python-machine-learning-model-as-a-web-service/
#Use pickle to export our model object as a binary, which can be used by the web service that will be created in the next step:
#import pickle
# pickle.dump(lm, open("../deploy/linearmodel.pkl","wb"))
#pickle.dump(lin_reg2, open("lin_reg2.pkl","wb"))

#https://developer.ibm.com/technologies/artificial-intelligence/tutorials/deploy-a-python-machine-learning-model-as-a-web-service/
import pickle
import flask
import os

app = flask.Flask(__name__)

#port = int(os.getenv("PORT", 9099))

#Import the model file that was created previously:model = pickle.load(open("linearmodel.pkl","rb"))
model= pickle.load(open("lin_reg2.pkl","rb"))
#pickle.dump(lin_reg2, open("lin_reg2.pkl","wb"))
@app.route("/")
def hello():
    return "this is where the machine learning model gets API-ed!"

#Add a route that will allow you to send a JSON body of features and will return a prediction:
@app.route('/index', methods=['POST'])
def predict():

    features = flask.request.get_json(force=True)['features']
    prediction = model.predict([features])[0,0]
    response = {'prediction': prediction}

    return flask.jsonify(response)

if __name__ == '__main__':
    app.run(debug=True)
 #    app.run(host='0.0.0.0', port=port)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09198038951108713, 'fraction_numerical': 0.012234066477758288, 'mean_word_length': 5.480326435441563, 'pattern_counts': {'":': 0, '<': 52, '<?xml version=': 0, '>': 52, 'https://': 18, 'lorem ipsum': 0, 'www.': 7, 'xml': 0}, 'pii_count': 1, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '82561', 'license': 'MIT', 'n_tokens_mistral': 6952, 'n_tokens_neox': 6622, 'n_words': 2556, 'path': 'Project2020/.ipynb_checkpoints/roughwork-checkpoint.ipynb', 'repo_name': 'ClodaghMurphy/MachineLearningandStatistics', 'size': '22233'}