<jupyter_start><jupyter_text>Introduction **Context**Cardiotocograms (CTGs) are a simple and cost accessible option to assess fetal health, allowing healthcare professionals to take action in order to prevent child and maternal mortality. The equipment itself works by sending ultrasound pulses and reading its response, thus shedding light on fetal heart rate (FHR), fetal movements, uterine contractions and more.**Data**This [dataset](https://www.kaggle.com/andrewmvd/fetal-health-classification) contains 2126 records of 21 features extracted from Cardiotocogram exams, which were then classified by three expert obstetritians into 3 classes:- Normal- Suspect- Pathological**Features**- 'baseline value' FHR baseline (beats per minute)- 'accelerations' Number of accelerations per second- 'fetal_movement' Number of fetal movements per second- 'uterine_contractions' Number of uterine contractions per second- 'light_decelerations' Number of light decelerations per second- 'severe_decelerations' Number of severe decelerations per second- 'prolongued_decelerations' Number of prolonged decelerations per second- 'abnormal_short_term_variability' Percentage of time with abnormal short term - variability- 'mean_value_of_short_term_variability' Mean value of short term variability- 'percentage_of_time_with_abnormal_long_term_variability' Percentage of time - with abnormal long term variability- 'mean_value_of_long_term_variability' Mean value of long term variability- 'histogram_width' Width of FHR histogram- 'histogram_min' Minimum (low frequency) of FHR histogram- 'histogram_max' Maximum (high frequency) of FHR histogram- 'histogram_number_of_peaks' Number of histogram peaks- 'histogram_number_of_zeroes' Number of histogram zeros- 'histogram_mode' Histogram mode- 'histogram_mean' Histogram mean- 'histogram_median' Histogram median- 'histogram_variance' Histogram variance- 'histogram_tendency' Histogram tendency**Target**- 'fetal_health' Tagged as 1 (Normal), 2 (Suspect) and 3 (Pathological)*Ayres de Campos et al. (2000) SisPorto 2.0 A Program for Automated Analysis of Cardiotocograms. J Matern Fetal Med 5:311-318* Loading and preparing the dataset<jupyter_code>library(readr)
install.packages("caret")
library(caret)
install.packages("rpart")
library(rpart)
install.packages("rpart.plot")
library(rpart.plot)
install.packages("randomForest")
library(randomForest)
install.packages("UBL")
library(UBL)
myfile <- "https://raw.githubusercontent.com/gobrac/datasets/main/fetal_health.csv"

fetal_health <- read_csv(myfile)
fetal_health$fetal_health =as.factor(fetal_health$fetal_health)

head(fetal_health)
#str(fetal_health)
summary(fetal_health)
set.seed(1)
trainIndex <- createDataPartition(fetal_health$fetal_health , p = .7,
                                  list = FALSE,
                                  times = 1)
Train <- data.frame(fetal_health [ trainIndex,])
Test <- data.frame(fetal_health [-trainIndex,])
Evaluate = function(Table){ 
  evaluation1 <- matrix(list(), nrow=3, ncol=3)

  rownames(evaluation1) <- c("class 1", "class 2", "class 3")
  colnames(evaluation1) <- c("Precision", "Recall", "F1 score")

  evaluation1[[1,1]] = Table[1] / sum(Table[,1])      # Precision: TP/(TP+FP)  
  evaluation1[[2,1]] = Table[5] / sum(Table[,2])      # Precision: TP/(TP+FP)  
  evaluation1[[3,1]] = Table[9] / sum(Table[,3])      # Precision: TP/(TP+FP)

  evaluation1[[1,2]] = Table[1] / sum(Table[1,])      # Recall: TP/(TP+FN)
  evaluation1[[2,2]] = Table[5] / sum(Table[2,])      # Recall: TP/(TP+FN)
  evaluation1[[3,2]] = Table[9] / sum(Table[3,])      # Recall: TP/(TP+FN)

  evaluation1[[1,3]] = 2*(evaluation1[[1,1]] * evaluation1[[1,2]] /  (evaluation1[[1,1]] + evaluation1[[1,2]]))   # F1 score: 2*(Precisio*Recall)/(Precision+Recall)  
  evaluation1[[2,3]] = 2*(evaluation1[[2,1]] * evaluation1[[2,2]] /  (evaluation1[[2,1]] + evaluation1[[2,2]]))   # F1 score: 2*(Precisio*Recall)/(Precision+Recall)
  evaluation1[[3,3]] = 2*(evaluation1[[3,1]] * evaluation1[[3,2]] /  (evaluation1[[3,1]] + evaluation1[[3,2]]))   # F1 score: 2*(Precisio*Recall)/(Precision+Recall)

  accuracy_Test <- sum(diag(Table)) / sum(Table)

  print(evaluation1)
  
  print(paste0("accuracy: ", accuracy_Test ))

}
dim(Train)
dim(Test)

prop.table(table(Train$fetal_health))
prop.table(table(Test$fetal_health))<jupyter_output><empty_output><jupyter_text>Treating unbalanced data<jupyter_code>table(Train$fetal_health)<jupyter_output><empty_output><jupyter_text>SMOTE algorithm [<NAME>, et al. (2002)](https://www.jair.org/index.php/jair/article/view/10302/24590) is ran to artificially generate new examples of the minority class (class 3) using the K-nearest neighbors ($K = 5$ by default):- Step 1: For each $x \in A$ (the minority class), the k-nearest neighbors of x are obtained by calculating the Euclidean distance between x and every other sample in set A.- Step 2: For each $x \in A$, n examples are randomly selected from its k-nearest neighbors, and they construct the set $A_1$ .- Step 3: For each example $x_k \in A_1$ (k=1, 2, 3…N), the following formula is used to generate a new example:$x' = x + rand(0, 1) * \mid x - x_k \mid$ in which rand(0, 1) represents the random number between 0 and 1. (in other words: computes a line between the minority data points and any of its neighbors and places a synthetic point somewhere the middle.)Furthermore, the majority class (class 1) is also under-sampled, leading to a  balanced dataset.<jupyter_code>set.seed(1)
Train = data.frame(Train)


Train <- SmoteClassif(fetal_health ~ ., Train, list("3" = 1.67, "1"= 0.1787))
table(Train$fetal_health)<jupyter_output><empty_output><jupyter_text>Decision Tree (rpart)By default, rpart uses gini impurity to select splits when performing classification$ G_i = 1 -   \sum_{k=1}^{k} (p_{i})^2 $Where$p_{i}$ is the ratio of class k instances among the training instances in the $i^{th}$ node (denotes the probability of an element being classified for each distinct class.)CART training algorithmthe algorithm searches for a single feature and its threashold that produce the purest subsets (weighted by their size). The cost function that the algorithm tries to minimize is:$J (k,t_k)=  \frac{m_{left}}{m} G_{left} + \frac{m_{right}}{m} G_{right}$Where:- $G_{left/right}$ measures the impurity of the left/right subset- $m_{left/right}$ is the number of instances in the left/right subsetOnce it has successfully split the training set in two, it splits the subsets using the same logic, then the sub-subsets and so on, recursively<jupyter_code>set.seed(1)

fit_0 <- rpart(fetal_health ~ ., method = "class", data = Train , control = rpart.control(cp = 0))

fit_0 
plotcp(fit_0)
rpart.plot(fit_0)

fit_0$cptable
predict_unseen0 <-predict(fit_0, Train, type = 'class')
table_mat0 <- table(Train$fetal_health, predict_unseen0) 
table_mat0<jupyter_output><empty_output><jupyter_text>Prune the treeThe tree pruning is based on the complexity parameter (cp) in rpart is the minimum improvement in the model needed at each node. It’s based on the cost complexity of the model defined as…$ \sum_{node=1}^{nodes} E_i + \lambda (n_{spits})$where:- $E_i$ is the misclassification at every terminal node.- $n_split$ is number of splits - $\lambda$ is a penalty term.The lambda is determined through cross-validation and not reported in R.The cp we see using printcp() is the scaled version of lambda over the misclassifcation rate of the overall data.<jupyter_code>Pruned_fit_0<-prune(fit_0,cp=0.014492754)

Pruned_fit_0
plotcp(Pruned_fit_0)
rpart.plot(Pruned_fit_0)
Pruned_fit_0$cptable
predict_unseen_pruned <-predict(Pruned_fit_0, Train, type = 'class')
table_mat1 <- table(Train$fetal_health, predict_unseen_pruned)
table_mat1


Evaluate(table_mat1)<jupyter_output>Precision Recall    F1 score 
class 1 0.8654709 0.9323671 0.8976744
class 2 0.9459459 0.8454106 0.8928571
class 3 0.9201878 0.9468599 0.9333333
[1] "accuracy: 0.908212560386473"<jupyter_text>Loss matrix<jupyter_code>lossmatrix <-matrix(c(0,1,1,3,0,1,10,5,0), byrow=TRUE, nrow=3, ncol = 3)
lossmatrix
set.seed(1)
fit_1 <- rpart(fetal_health ~ ., method = "class", data = Train, parms = list(loss = lossmatrix) )

fit_1 
printcp(fit_1)
plotcp(fit_1)
rpart.plot(fit_1)


predict_unseen1 <-predict(fit_1, Train, type = 'class')
table_1 <- table(Train$fetal_health, predict_unseen1) 
table_1
Evaluate(table_1)<jupyter_output>Precision Recall    F1 score 
class 1 0.9642857 0.7826087 0.864    
class 2 0.8518519 0.8888889 0.8699764
class 3 0.8565401 0.9806763 0.9144144
[1] "accuracy: 0.884057971014493"<jupyter_text>Evaluate on test set<jupyter_code>predict_unseen1 <-predict(fit_1, Test, type = 'class')
table_test <- table(Test$fetal_health, predict_unseen1) 
table_test
Evaluate(table_test)<jupyter_output>Precision Recall    F1 score 
class 1 0.9886686 0.703629  0.8221437
class 2 0.4716981 0.8522727 0.6072874
class 3 0.4193548 1         0.5909091
[1] "accuracy: 0.748427672955975"<jupyter_text>Random Forest<jupyter_code>set.seed(1)

Train_df=data.frame(Train)
RF0 <- randomForest(fetal_health ~ ., data = Train_df, type ='class', ntree = 500, importance = TRUE) 
plot(RF0,  main="Random forest")
legend("topright", colnames(RF0$err.rate),col=1:4,cex=0.8,fill=1:4)



#importance(RF0)
table_matRF <- table(Train$fetal_health, RF0$predicted)
table_matRF
Evaluate(table_matRF)

set.seed(12)
my.mtry <- tuneRF(Train[-22],Train$fetal_health, ntreeTry=500,
stepFactor=1.5,improve=0.001, trace=TRUE, plot=TRUE)
my.best <- my.mtry[which.min(my.mtry[,2]),1]
RF1 <- randomForest(fetal_health ~ ., data = Train_df, type ='class', ntree = 500, mtry = my.best, importance = TRUE)

RF1

table_matRF1 <- table(Train$fetal_health, RF1$predicted)
table_matRF1

Evaluate(table_matRF1)
#Evaluate variable importance

varImpPlot(RF1, sort=T, type = 1)
varImpPlot(RF1, sort=T, type = 2)<jupyter_output><empty_output><jupyter_text>Evaluate on test set<jupyter_code>predict_unseenRF <-predict(RF1, Test, type = 'class')
table_testRF <- table(Test$fetal_health, predict_unseenRF) 
table_testRF
Evaluate(table_testRF)<jupyter_output>Precision Recall    F1 score 
class 1 0.986456  0.8810484 0.9307774
class 2 0.6341463 0.8863636 0.7393365
class 3 0.7285714 0.9807692 0.8360656
[1] "accuracy: 0.889937106918239"
{'fraction_non_alphanumeric': 0.12082237162623015, 'fraction_numerical': 0.06128812238136997, 'mean_word_length': 5.673602080624187, 'pattern_counts': {'":': 0, '<': 64, '<?xml version=': 0, '>': 34, 'https://': 3, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '194238', 'license': 'Unlicense', 'n_tokens_mistral': 4039, 'n_tokens_neox': 3512, 'n_words': 965, 'path': 'Projects/Statistical Learning - Prof. Gottard.ipynb', 'repo_name': 'gobrac/Portfolio---data-Analyst', 'size': '10270'}