<jupyter_start><jupyter_text>Classifying Driver Type with Brake Events By <NAME>, <NAME>, <NAME>, and <NAME> ________________________________**Table of contents**    * [Problem Statement](problemStatement)    * [Exploratory Data Analysis](eda)* [Modeling](ml)    * [Data and Model Export](export)* [Conclusion](conclusion)______________________ Problem StatementThe service bays at dealerships have seen an increase in warranty claims related to brakes.  Using historical telematics data of known driver types, can we classify the driving style of customers making warranty claims?________ Exploratory Data Analysis  ** Action Required ** 1. In the code cell below, click **Insert to code** the **historical_brake_events.csv** as an R Datafame2. Rename the generated **df.data.1** variable to **brakeEventDF****Note:** You are reading the csv file from the project data assets<jupyter_code>#add code to read historical_brake_events.csv as an R Datafame<jupyter_output><empty_output><jupyter_text>We see VINs, the type or classification of the brake event, and then a series of columns related to the brake event itself.   Summary StatisticsLet's begin exploring the data by looking at some summary statistics of these events by both type and road type.<jupyter_code>library(magrittr)
library(dplyr)
print("Summary Statistics by Event Type")
group_by(brakeEventDF, type) %>% summarise(avg_braketime = mean(brake_time_sec), avg_brakedistance = mean(brake_distance_ft), avg_brakescore = mean(braking_score), abs_events = sum(abs_event))

print("Summary Statistics by Event Type and Road Type")
aggDF <- group_by(brakeEventDF, type, road_type) %>% summarise(avg_braketime = mean(brake_time_sec), avg_brakedistance = mean(brake_distance_ft), avg_brakescore = mean(braking_score), abs_events = sum(abs_event))

aggDF<jupyter_output><empty_output><jupyter_text>Looks like aggressive drivers have lower brake times, distances, and scores.  Distracted drivers have more ABS events.  Quality drivers are on the other side of the spectrum.   VisualizationWe can see these relationships visually using the open source R package, ggplot2.  Let's examine the following three relationships:* Brake Time by Type* Brake Distance by Braking Score* ABS Events by Type and Road Type<jupyter_code>library(ggplot2)
options(repr.plot.width = 12, repr.plot.height = 3)

ggplot(brakeEventDF, aes(x = brake_time_sec, color = type, fill = type)) + 
    geom_density(alpha = 0.5) +
    labs(x = "Braking Time (seconds)", y = "Observation Density", title = "Distribution of Brake Time by Type") +
    theme_minimal()

ggplot(sample_frac(brakeEventDF, .33), aes(x = brake_distance_ft, y = braking_score)) + 
    geom_point(aes(shape = road_type, color = type), size = 2) +
    scale_shape_manual(values=c(3, 5, 8)) +
    geom_point(color = 'black', size = 0.35, aes(shape = road_type)) +
    labs(x = "Braking Distance (feet)", y = "Braking Score", title = "Braking Score by Distance (ft)") +
    theme_minimal()

ggplot(aggDF, aes(x = road_type, y = abs_events)) + 
    geom_bar(aes(fill = type), stat = 'identity') + 
    coord_flip() +
    labs(x = "# of ABS Events", y = "Road Type", title = "ABS Events by Road Type and Event Type") +
    theme_minimal()<jupyter_output><empty_output><jupyter_text>After visually inspecting the data, we see some clear grouping along the lines of event type, road type, and number of ABS events.  There is also an obvious linear relationship between brake score and brake time.  This historical data is clean enough to build a model from.__________ Model Training with Caret PackageBy using the Caret package to build the model, and saving it to the ML repository, you will be able to evaluate and score the saved R model.  See documentation on the Caret Package and Saving R modelsThe caret package (short for _C_lassification _A_nd _RE_gression _T_raining) is a set of functions that attempt to streamline the process for creating predictive models. The package contains tools for:    * data splitting    * pre-processing    * feature selection    * model tuning using resampling    * variable importance estimationas well as other functionality.  Example using `train()` functionWe can train a decision tree model on the historical brake event data.  It will learn the relationship between the various quantitative variables and the type of brake event, allowing us to classify new records as they come in.  Split data into train and test sets<jupyter_code>suppressWarnings(suppressMessages(library(caTools)))
suppressWarnings(suppressMessages(library(randomForest)))
suppressWarnings(suppressMessages(library(caret)))
set.seed(22)

inTraining <- createDataPartition(brakeEventDF$type, p = .70, list = FALSE)
trainingDF <- brakeEventDF[ inTraining,]
testingDF  <- brakeEventDF[-inTraining,]

## Check dimensions, should add up to 2100
paste("Rows in training set: ", dim(trainingDF)[1])
paste("Rows in test set: ", dim(testingDF)[1])<jupyter_output><empty_output><jupyter_text>Select features, train model, evaluate accuracy<jupyter_code>## Preserve VINs to add on after modeling
vins <- trainingDF$VIN

## Select columns for modeling
trainingDF <- select(trainingDF, type, brake_time_sec, brake_distance_ft, road_type, braking_score, 
                 brake_pressure20pct, brake_pressure40pct, brake_pressure60pct,
                 brake_pressure80pct, brake_pressure100pct, abs_event, travel_speed)

# The function trainControl can be used to specifiy the type of resampling
# Here, three separate 10-fold cross-validations are used as the resampling scheme
fitControl <- trainControl(## 10-fold CV
                           method = "repeatedcv",
                           number = 10,
                           ## repeated 3 times
                           repeats = 3)


## Using `caret` package
brakeEventModel <- train(type ~ .,
                         data = trainingDF,
                         method = "rf",
                         ntree = 50,
                         trControl=fitControl,
                         proximity = TRUE)

print("Confusion Matrix for Testing Data:")
table(predict(brakeEventModel, select(testingDF, -type)), testingDF$type)
brakeEventModel<jupyter_output><empty_output><jupyter_text>Model ExportSave model in the **WSL RStudio directory** for use in our Shiny app. When we save to the file system, we will not be able to take advantage of the built-in model deployment capabilties. However, we use this option  just for demo purposes because we can quickly integrate it with the Shiny application. To integrate with the model that has been saved in a WSL repository, we will need to deploy it for online scoring in **Deployment Manager** and invoke it from Shiny via REST API.   ** Action Required ** 1. Take note of the directory in RStudio where the *brakeEventModel.rds* file is saved<jupyter_code>saveRDS(object = brakeEventModel, file = "../rstudio/demoBrakeEvents/brakeEventModel.rds")<jupyter_output><empty_output><jupyter_text>Verify that the model has been saved.<jupyter_code>print(system("pwd", intern = TRUE))
print(system("ls -l ../rstudio/demoBrakeEvents", intern = TRUE))<jupyter_output><empty_output><jupyter_text>The model has successfully been saved to File System. Now **save the model to Watson Studio Local** so that it shows up in your project assets.  Models saved in WSL can be scored and evaluated.<jupyter_code>### Save Model in the WSL repository

suppressWarnings(suppressMessages(library(modelAccess)))
suppressWarnings(suppressMessages(library(jsonlite)))

saveModel(model = brakeEventModel, name = "BrakeEventClassifier", test_data=testingDF)<jupyter_output><empty_output><jupyter_text>** Action Required ** Load the saved model from the WSL repository for scoring.  Review Load caret model in RTake note of the model path in the code cell below, compare it to the "path" displayed after executing the previous code cell to save the model in WSL<jupyter_code>modelPath <- paste(Sys.getenv("DSX_PROJECT_DIR"),"/models/BrakeEventClassifier/1/model",sep="")
savedModel<-readRDS(modelPath)
# payload for scoring 

scoringDF <- data.frame(
      brake_time_sec = 40,
      brake_distance_ft = 120,
      road_type = as.factor("highway"),
      braking_score = 100,
      brake_pressure20pct = 1,
      brake_pressure40pct = 1,
      brake_pressure60pct = 0,
      brake_pressure80pct = 0,
      brake_pressure100pct = 0,
      abs_event = 1,
      travel_speed = 20)
    
predictions <- predict(savedModel, scoringDF)
probabilities <- predict(savedModel, scoringDF, type="prob")
classes <- colnames(probabilities)

output <- list(classes, unname(probabilities, force=FALSE), predictions)
names(output) <- list("classes", "probabilities", "predictions")
json_output <- toJSON(output)
json_output<jupyter_output><empty_output><jupyter_text>** Action Required **  Verify the two different locations of the saved model1. Go to the Models section of the project, verify that the BrakeEventClassifier model is there2. Open RStudio:     2.1 Verify that the brakeEventsModel.rds is saved in the **demoBrakeEvents** folder     2.2 Open demoBrakeEvents\server.R and and click "Run App" to run the Shiny App  Optional ExerciseThis exercise assumes you have:* Deployed this project into WML* Created an online deployment for the BrakeEventClassifier model**Action Required*** Copy the generated code in the online deployment* Edit the code cell below to reolace the URL and the value of the Authorization Bearer token with those of your own deployment* Execute the code cell below<jupyter_code># payload for scoring 

scoringDF <- data.frame(
      brake_time_sec = 40,
      brake_distance_ft = 120,
      road_type = as.factor("highway"),
      braking_score = 100,
      brake_pressure20pct = 1,
      brake_pressure40pct = 1,
      brake_pressure60pct = 0,
      brake_pressure80pct = 0,
      brake_pressure100pct = 0,
      abs_event = 1,
      travel_speed = 20)
    
suppressWarnings(suppressMessages(library(jsonlite)))

## Use generated code from model API
curl_left <- "curl -k -X POST https://172.16.31.10/dmodel/v1/r-lab-test-rel/rscript/brakeevent-online/score -H \'Authorization: <KEY>\' -H \'Cache-Control: no-cache\' -H \'Content-Type: application/json\' -d \'{\"args\":{\"input_json\":"
curl_right <- "}}\'"
    
## Convert your dataframe for scoring to JSON that can be sent in the request body via REST
request_body <- toJSON(scoringDF)
    
## Make request by passing the curl command with the JSON-formatted request body to the system
response <- system(paste0(curl_left, request_body, curl_right), intern = T)
    
## Parse the response from the API back into an R dataframe
prediction <- as.data.frame(fromJSON(unlist(fromJSON(response)[1])))

results <- list(prediction = prediction, response = response)
results<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.0867318693485883, 'fraction_numerical': 0.011441225318324415, 'mean_word_length': 4.532924961715161, 'pattern_counts': {'":': 2, '<': 70, '<?xml version=': 0, '>': 48, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 1, 'substrings_counts': 1, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '363652', 'license': 'Apache-2.0', 'n_tokens_mistral': 3282, 'n_tokens_neox': 3057, 'n_words': 1123, 'path': 'jupyter/DriverClassification.jupyter-py35.ipynb', 'repo_name': 'krondor/wsl-workshop', 'size': '11380'}