<jupyter_start><jupyter_text>Exercises1. Download boston dataset from eLearn@USM<jupyter_code>boston_data <- read.csv("boston.data.csv", sep=",",  stringsAsFactors=TRUE)<jupyter_output><empty_output><jupyter_text>2. Load the datase<jupyter_code>summary(boston_data)<jupyter_output><empty_output><jupyter_text>3. Detect any missing value in the dataset    - Answer: from the summary of the data we can see there are missing values on `MEDV` column    4. Detect any outlier in column DIS using boxplot<jupyter_code>library(ggplot2)
library(scales)
library(mgcv)
class(boston_data$DIS_df)
ggplot(boston_data, aes(x=1, y=DIS)) + 
  geom_boxplot() +
  scale_x_continuous(breaks = NULL) + #removes the tick markers from the x axis
  theme(axis.title.x = element_blank())<jupyter_output><empty_output><jupyter_text>Answer: As shown by the boxplot above, there is one outlier in DIS column  5. Write a programming function for IQR rule. The function should accept first quartile and third quartile as arguments and return the lower and upper bounds as a vector. Use the function to detect any outlier in column LSTAT $$IQR = Q3 - Q1$$$Q3 : 75^{th}\ percentile$$Q1 : 25^{th}\ percentile$<jupyter_code># defining and calculating Q1
Q1 <- quantile(boston_data$LSTAT, 0.25)
Q1
# defining and calculating Q3
Q3 <- quantile(boston_data$LSTAT, 0.75)
Q3<jupyter_output><empty_output><jupyter_text>$$Upper\ bound\ =\ Q3 + (1.5\ x\ IQR)$$$$Lower\ bound\ =\ Q1 - (1.5\ x\ IQR)$$<jupyter_code>iqr_bound <- function(Q1, Q3){
    iqr <- (Q3 - Q1)
    low_bound <- (Q1 - (1.5*iqr))
    up_bound <- (Q3 + (1.5*iqr))
cat(paste("iqr = ", iqr, "\nlower-bound = ", low_bound, 
          "\nupper-bound = ", up_bound))
}
iqr_bound(Q1, Q3)<jupyter_output>iqr =  8.1375 
lower-bound =  -5.32875 
upper-bound =  27.22125<jupyter_text>6. Examine the relationship between attribute AGE and attribute TAX. Determine if it is a positive or negative correlation or no correlation<jupyter_code>ggplot(boston_data, aes(x=AGE, y=TAX)) + geom_point() + ylim(0, 1000) + geom_smooth()<jupyter_output>`geom_smooth()` using method = 'loess' and formula 'y ~ x'<jupyter_text>Answer : There is no clear correlation between attribute AGE and TAX7. Visualize the relationship between AGE and TAX and fit a linear line through the data. Observe the slope of the linear line<jupyter_code>ggplot(boston_data, aes(x=AGE, y=TAX)) + geom_point() + ylim(0,1000) + stat_smooth(method='lm')<jupyter_output>`geom_smooth()` using formula 'y ~ x'
{'fraction_non_alphanumeric': 0.12313883299798793, 'fraction_numerical': 0.030181086519114688, 'mean_word_length': 5.341836734693878, 'pattern_counts': {'":': 0, '<': 33, '<?xml version=': 0, '>': 26, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '136012', 'license': 'MIT', 'n_tokens_mistral': 967, 'n_tokens_neox': 894, 'n_words': 248, 'path': 'Courses (USM)/CDS501 Principles & Practices of Data Science/Lab 3/Lab 3.ipynb', 'repo_name': 'ramdhanhdy/Courses', 'size': '2485'}