<jupyter_start><jupyter_text>A Machine Learning Project may not be linear, but it has a number of well known steps:1.Define Problem.2.Prepare Data.3.Evaluate Algorithms.4.Improve Results.5.Present Results.In this section, we are going to work through a small machine learning project end-to-end.Here is an overview of what we are going to cover:1.Installing the Python and SciPy platform.2.Loading the dataset.3.Summarizing the dataset.4.Evaluating some algorithms.5.Making some predictions. 1.Downloading, Installing and Starting Python SciPy<jupyter_code># from xml.dom import minidom
import numpy as np
import pandas
import csv
from pandas.plotting import scatter_matrix
import matplotlib.pyplot as plt
from sklearn import model_selection
from sklearn.metrics import classification_report
from sklearn.metrics import confusion_matrix
from sklearn.metrics import accuracy_score
from sklearn.svm import SVC<jupyter_output><empty_output><jupyter_text>2. Load The Data<jupyter_code>#Load dataset
url ='dataset/LaneClosure/L10130/2 s interval(including2hops)/1,2,3 close/L30.csv'
names = ["Time","Edge ID","Edge Length","NumberOfLane","Lane Name","Jam Length","Density","Mean Speed","Mean Occupancy","Flow","Road State(basedOnJamLength)","Road State(basedOnFlow)"]
dataset = pandas.read_csv(url, names=names,skiprows=1)
plt.scatter(dataset['Flow'], dataset['Mean Speed'])
# class distribution
dataset.groupby('Road State(basedOnFlow)').head()

abnormalcase = dataset.loc[dataset['Road State(basedOnFlow)']==1];
normalcase = dataset.loc[dataset['Road State(basedOnFlow)']==0];

plt.scatter(normalcase['Flow'], normalcase['Mean Speed']);
plt.xlim(0,30); plt.ylim(0,1.1)
plt.figure(); plt.scatter(abnormalcase['Flow'], abnormalcase['Mean Speed']);<jupyter_output><empty_output><jupyter_text>3.In this step we are going to take a look at the data a few different ways:Dimensions of the dataset.Peek at the data itself.Statistical summary of all attributes.Breakdown of the data by the class variable.<jupyter_code># descriptions
print(dataset.describe())
# class distribution
print(dataset.groupby('Road State(basedOnFlow)').size())<jupyter_output>Road State(basedOnFlow)
0    3505
1    1896
dtype: int64<jupyter_text>4. Evaluate Some AlgorithmsNow it is time to create some models of the data and estimate their accuracy on unseen data.Here is what we are going to cover in this step:Separate out a validation dataset.Set-up the test harness to use 10-fold cross validation.Build 5 different models to predict species from flower measurementsSelect the best model.<jupyter_code># Split-out validation dataset
array = dataset.values
X = array[:,[7,9]]

Y = array[:,11]
Y=Y.astype('int')

validation_size = 0.30
seed = 7
X_train, X_validation, Y_train, Y_validation = model_selection.train_test_split(X, Y, test_size=validation_size, random_state=seed)

print(X)
# Test options and evaluation metric
seed = 7
scoring = 'accuracy'
# Spot Check Algorithms
models=[]

svc = SVC(kernel='rbf', C=1,gamma='auto').fit(X_train, Y_train)

models.append(('SVM', svc))
# evaluate each model in turn
results = []
names = []
for name, model in models:
    kfold = model_selection.KFold(n_splits=10, random_state=seed)
    #kfold = model_selection.KFold(n_splits=10)
    cv_results = model_selection.cross_val_score(svc, X_train, Y_train, cv=kfold, scoring=scoring)
    results.append(cv_results)
    names.append(name)
    
    msg = "%s: %f (%f)" % (name, cv_results.mean(), cv_results.std())
    print(msg)<jupyter_output>SVM: 0.729630 (0.021058)<jupyter_text>5.Make Predictions<jupyter_code># Make predictions on validation dataset
predictions = svc.predict(X_validation)
print('Accuracy :',accuracy_score(Y_validation, predictions))
cm=confusion_matrix(Y_validation, predictions)

plt.clf()
plt.imshow(cm, interpolation='nearest', cmap=plt.cm.Wistia)
classNames = ['Normal','Abnormal']
plt.title('Normal or Abnormal Confusion Matrix')
plt.ylabel('True label')
plt.xlabel('Predicted label')
tick_marks = np.arange(len(classNames))
plt.xticks(tick_marks, classNames, rotation=45)
plt.yticks(tick_marks, classNames)
s = [['TN','FP'], ['FN', 'TP']]
for i in range(2):
    for j in range(2):
        plt.text(j,i, str(s[i][j])+" = "+str(cm[i][j]))
plt.show()

# create a mesh to plot in
x_min, x_max = X[:, 0].min() - 0, X[:, 0].max() + 0
y_min, y_max = X[:, 1].min() - 0, X[:, 1].max() + 0
h = (x_max / x_min)/100
xx, yy = np.meshgrid(np.arange(x_min, x_max, h),
 np.arange(y_min, y_max, h))
plt.subplot(1, 1, 1)
Z = svc.predict(np.c_[xx.ravel(), yy.ravel()])
Z = Z.reshape(xx.shape)
plt.contourf(xx, yy, Z, cmap=plt.cm.Paired, alpha=0.8)
plt.scatter(X[:, 0], X[:, 1], c=Y, cmap=plt.cm.Paired)
plt.xlabel('Mean Speed')
plt.ylabel('Flow')
plt.xlim(xx.min(), xx.max())
plt.title('SVC with RBF kernel')
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13817427385892117, 'fraction_numerical': 0.020954356846473027, 'mean_word_length': 6.174107142857143, 'pattern_counts': {'":': 0, '<': 19, '<?xml version=': 0, '>': 19, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 1}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '406483', 'license': 'Apache-2.0', 'n_tokens_mistral': 1752, 'n_tokens_neox': 1656, 'n_words': 440, 'path': 'Traffic Anomaly Classification by Support Vector Machine with Radial Basis Function on Chula-SSS Urban Road Network/AnomalyDetection.ipynb', 'repo_name': 'Sinadalee/Smart-Mobility-Chula', 'size': '4820'}