<jupyter_start><jupyter_text>Defining the question  a) Specifying The Data Analytic question> To identify who is more likely to click on ad on a blog site  b) Defining the Metrics of success> The successful EDA should have:* Find and deal with outliers, anomalies, and missing data within the dataset.* Perform  univariate and bivariate analysis.* From your insights provide a conclusion and recommendation.  d) Recording the Experimental Design> 1. Reading the data> 2. Checking the data> 3. External Data Source Validation> 4. Tidying the dataset> 5. Univariate Analysis> 6. Bivariate Analysis e) Data RelevanceThe data is relevant since it has important columns:* Daily.Time.Spent.on.Site - The time spent by viewers on the blog site* Age - The age of the blog readers* Area.Income * Daily.Internet.Usage* Ad.Topic.Line* City* Male* Country* Timestamp* Clicked.on.Ad<jupyter_code>#Installing necessary packages in R
install.packages("data.table", dependencies=TRUE)
#Installing tidyverse
install.packages("tidyverse")
library(tidyverse)
library(lubridate)#work with dates
# load the ggplot2 library for plotting
library(ggplot2)
options(stringsAsFactors = FALSE)
install.packages("ggplot2")
library(ggplot2)
library('scales') # visualization
library('grid') # visualisation


library('dplyr') # data manipulation
library('readr') # data input
library('tibble') # data wrangling
library('tidyr') # data wrangling
library('stringr') # string manipulation
library('forcats') # factor manipulation<jupyter_output>Installing package into ‘/usr/local/lib/R/site-library’
(as ‘lib’ is unspecified)

Installing package into ‘/usr/local/lib/R/site-library’
(as ‘lib’ is unspecified)

── [1mAttaching packages[22m ─────────────────────────────────────── tidyverse 1.3.0 ──

[32m✔[39m [34mggplot2[39m 3.3.2     [32m✔[39m [34mpurrr  [39m 0.3.4
[32m✔[39m [34mtibble [39m 3.0.3     [32m✔[39m [34mdplyr  [39m 1.0.2
[32m✔[39m [34mtidyr  [39m 1.1.1     [32m✔[39m [34mstringr[39m 1.4.0
[32m✔[39m [34mreadr  [39m 1.3.1     [32m✔[39m [34mforcats[39m 0.5.0

── [1mConflicts[22m ────────────────────────────────────────── tidyverse_conflicts() ──
[31m✖[39m [34mdplyr[39m::[32mfilter()[39m masks [34mstats[39m::filter()
[31m✖[39m [34mdplyr[39m::[32mlag()[39m    masks [34mstats[39m::lag()


Attaching package: ‘lubridate’


The following objects are masked from ‘package:base’:

    date, intersect, setdiff, union


Installing package into ‘/usr/local/lib/R/site-library’
(as ‘li[...]<jupyter_text>Reading the dataset<jupyter_code>df <- read.csv(file ='advertising.csv', fileEncoding="UTF-8-BOM", na.strings = '..')

#Previewing the first six entries
head(df)
#Previewing the last six columns
tail(df)
#Checking for column names and datatypes

str(df)
#this will  give us the shape of the dataframe
dim(df)<jupyter_output><empty_output><jupyter_text>The dataset has 10 columns and 1000 rows<jupyter_code>class(df)
# get column names
colnames(df)
#Checking for the data summaries
summary(df)<jupyter_output><empty_output><jupyter_text>Validity<jupyter_code>#Replacing column headings for better and faster analysis
df<-df %>% 
  rename(
    time_on_site = Daily.Time.Spent.on.Site,
    age = Age,
    income = Area.Income,
    internet_usage = Daily.Internet.Usage,
    topic_line = Ad.Topic.Line,
    city = City,
    country = Country,
    gender = Male,
    timestamp = Timestamp,
    clicked_on_ad = Clicked.on.Ad
      )<jupyter_output><empty_output><jupyter_text>Completeness<jupyter_code>#identifying missing data using is.na()function
is.na(df)
#Checking number of missing values
length(which(is.na(df)))<jupyter_output><empty_output><jupyter_text>The data is complete since there are no missing values  Consistency<jupyter_code># Finding the duplicated rows in the dataset df 
# and assign to a variable duplicated_rows below
duplicated_rows <- df[duplicated(df),]
duplicated_rows<jupyter_output><empty_output><jupyter_text>Our data is consistent since we have no duplicated values  Uniformity<jupyter_code>#Checking if we  have correct data types  for our variables
str(df)<jupyter_output>'data.frame':	1000 obs. of  10 variables:
 $ time_on_site  : num  69 80.2 69.5 74.2 68.4 ...
 $ age           : int  35 31 26 29 35 23 33 48 30 20 ...
 $ income        : num  61834 68442 59786 54806 73890 ...
 $ internet_usage: num  256 194 236 246 226 ...
 $ topic_line    : chr  "Cloned 5thgeneration orchestration" "Monitored national standardization" "Organic bottom-line service-desk" "Triple-buffered reciprocal time-frame" ...
 $ city          : chr  "Wrightburgh" "West Jodi" "Davidton" "West Terrifurt" ...
 $ gender        : int  0 1 0 1 0 1 0 1 1 1 ...
 $ country       : chr  "Tunisia" "Nauru" "<NAME>" "Italy" ...
 $ timestamp     : chr  "2016-03-27 00:53:11" "2016-04-04 01:39:02" "2016-03-13 20:35:42" "2016-01-10 02:31:19" ...
 $ clicked_on_ad : int  0 0 0 0 0 0 0 1 0 0 ...<jupyter_text>Seperating Timestamp to date variable and time variables.This will be of importance in the analysis<jupyter_code>#Changing male and clicked on Ad variable from integer to a factor
df$gender <- factor(df$gender, labels = c("M", "F"))
df$clicked_on_ad <- factor(df$clicked_on_ad, labels = c('yes', 'no'))

#Seperating date from the timestamp variable
#assigning date into date varaible with date data type
df$date <- as.Date(df$timestamp)
#Displaying the first six
head(df)
#Seperating time from the timestamp variable
#assigning time into time varaible with character data type
df$time <- format(as.POSIXct(df$timestamp), format = "%H:%M:%S") 
#Previewing first six
head(df)
#Splitting the date column into year, month and day
df_date<-df %>%
  dplyr::mutate(year = lubridate::year(date), 
                month = lubridate::month(date), 
                day = lubridate::day(date))
#Deleting the timestampcolumn
df_date<-df_date %>% select(-timestamp)
#Changing into factors
df_date$year<- factor (df_date$year)
df_date$month<- factor(df_date$month)
df_date$day <- factor(df_date$day)
#Previewing the first six rows
head(df_date)<jupyter_output><empty_output><jupyter_text>Checking for outliers  Numerical Variables<jupyter_code>#Checking for outliers in the Daily.Time.Spent.on.Site variable
boxplot(df$time_on_site,
main = "Daily.Time.Spent.on.Site boxplot",
xlab = "Count",
ylab = "Time Spent",
col = "orange",
border = "brown",
horizontal = TRUE,
notch = TRUE)
#Checking for outliers in the Age variable
boxplot(df$age,
main = "Age boxplot",
xlab = "Age",
ylab = "Count",
col = "orange",
border = "brown",
horizontal = TRUE,
notch = TRUE)
#Checking for outliers in the Area.income variable
boxplot(df$income,
main = "income boxplot",
xlab = "income",
ylab = "Count",
col = "orange",
border = "brown",
horizontal = TRUE,
notch = TRUE)
#Checking for outliers in the internet usage variable
boxplot(df$internet_usage,
main = "internet usage boxplot",
xlab = "internet usage in seconds",
ylab = "Count",
col = "orange",
border = "brown",
horizontal = TRUE,
notch = TRUE)<jupyter_output><empty_output><jupyter_text>Income variable was the only column variable with outiers. We will keep the outliers since they will be needed in the analysis  Categorical variables<jupyter_code>df<- dplyr::mutate(df, id = row_number())
df_date<- dplyr:: mutate(df_date, id= row_number())
boxplot(id ~ gender , data= df)
boxplot(id~clicked_on_ad, data = df)
boxplot(id ~ year, data = df_date)
boxplot(id ~ month, data = df_date)
boxplot(id ~ day, data = df_date)<jupyter_output><empty_output><jupyter_text>There are  no outliers in the categorical variables  Univariate Analysis<jupyter_code>#Selecting the numeric variables
df_num<-df_date %>% select_if(is.numeric)
head(df_num)
#Selecting factor variables
df_factor<- df_date%>% select_if(is.factor)
head(df_factor)
#Selecting character variables
df_char <- df_date %>% select_if(is.character)
head(df_char)<jupyter_output><empty_output><jupyter_text>Measure Of central Tendancy<jupyter_code>#Finding the mean of numerical variables grouped on clicked on ad
mean_df <-df %>%
  group_by(clicked_on_ad) %>%
  summarise_if(is.numeric, mean)
mean_df
#Finding the mean of numerical variables grouped on clicked on ad
median_df <-df %>%
  group_by(clicked_on_ad) %>%
  summarise_if(is.numeric ,median)
median_df<jupyter_output><empty_output><jupyter_text>Measures of Dispersion<jupyter_code>#Calculating the std of the variables grouped on clicked on ad
sd_df <-df %>%
  group_by(clicked_on_ad) %>%
  summarise_if(is.numeric ,sd)
sd_df
#Calculating variance of the variables grouped on clicked on ad
var_df <-df %>%
  group_by(clicked_on_ad) %>%
  summarise_if(is.numeric ,var)
var_df<jupyter_output><empty_output><jupyter_text>Categorical Univariate Analysis  Frequency Tables<jupyter_code>#Fetching the gender column
gender<-df$gender
#Fetching the cicked on ad column
clicked<-df$clicked_on_ad
#Fetching the month 
month<- df_date$month
#Fetching the day
day<-df_date$day
#Fetching the country column
country<- df$country
#Fetching the city column
city<- df$city

#Finding the frequency tables
#Finding the frequency table for gender
gender_frequency<-table(gender)
gender_frequency
cat('***************')
#Finding the frequency table for clicked on ad
clicked_frequency<- table(clicked)
clicked_frequency
cat('*******************')
#Finding the frequency table for months
months_frequency <- table(month)
months_frequency
cat('************')
#Finding the frequency table for days
day_frequency <- table(day)
day_frequency
#Finding the frequency of country
country_frequency<- table(country)
country_frequency
#Finding the frequency of city
city_frequency<- table(city)
city_frequency<jupyter_output><empty_output><jupyter_text>Bar charts<jupyter_code>barplot(gender_frequency)
# Change barplot fill colors by groups
p<-ggplot(df, aes(x=clicked_on_ad, y=id, fill=clicked_on_ad)) +
  geom_bar(stat="identity")+theme_minimal()
p
library(ggplot2)
# Change barplot fill colors by groups
p<-ggplot(df, aes(x=month, y=id, fill=month)) +
  geom_bar(stat="identity")+theme_minimal()
p
# Change barplot fill colors by groups
p<-ggplot(df, aes(x=day, y=id, fill=day)) +
  geom_bar(stat="identity")+theme_minimal()
p<jupyter_output><empty_output><jupyter_text>Frequency Graphs<jupyter_code>p7 <- ggplot(df, aes(x = age)) +
        geom_histogram()
p7
#Computing the range of Age variable
age_range<- range(df$age)
age_range
p7 <- ggplot(df, aes(x = income)) +
        geom_histogram()
p7
income_range<- range(df$income)
income_range
p7 <- ggplot(df, aes(x = time_on_site)) +
        geom_histogram()
p7
time_range<- range(df$time_on_site)
time_range
p7 <- ggplot(df, aes(x = internet_usage)) +
        geom_histogram()
p7
usage_range <- range(df$internet_usage)
usage_range<jupyter_output><empty_output><jupyter_text>**Categorical Summaries:*** Gender - is a factor variable. There are more males than females in the variable* Clicked on ad - is a factor variabe. There are more viewers who did not click on the ad as compared to those who actually cicked on the ad* Month - is a factor variable. There were more clicks in the 2,3 and 5 month as compared to other months. There was a significant decline in the seventh month.* Day - is a factor variable. The 31st recorded the lowest number of clicks on ad**Numerical Summaries:*** Age- is a numerical variable. The age range of viewers is between 19 and 61. While between 30-40 year olds are the most viewers.* Income - is a numerical variable. The income range is between 13,996.5 - 79,484.8. Most viewers earn between 50,000 to 70,000.*  Time_on_site - is a numerical variable. The viewers time on site ranged between 32.6 - 91.43 seconds. Most viewers spend 70 to 90 seconds on the site.* Internet Usage - is a numerical variable. The range of internet usage of viewers is between 104.78-269.96 seconds.   Bivariate Analysis<jupyter_code>#Calculating the correlation summaries of numerical variables
install.packages("ggcorrplot")
library(ggcorrplot)

corr <- round(cor(df_num), 1)
corr<jupyter_output>Installing package into ‘/usr/local/lib/R/site-library’
(as ‘lib’ is unspecified)<jupyter_text>Time_on_site against clicked on ad<jupyter_code>JitterPlot = qplot(clicked_on_ad, time_on_site, data=df, geom=c("boxplot", "jitter"), 
                   main="Time_on_site versus clicked_on_ad",xlab="clicked on ad", ylab="Time on site")
JitterPlot<jupyter_output><empty_output><jupyter_text>The viewers who were more likely to click on ad, spent more time on the blog site as compared to those who were not likely to click on site  Age against clicked on ad<jupyter_code>df %>% select(age,time_on_site ) %>% ggplot(aes(factor(age),time_on_site)) + geom_boxplot() + xlab('Age')+ 
      labs(title = "Time on site per age")<jupyter_output><empty_output><jupyter_text>Viewers between age 19 and age 36 spend more time on the site as compared to 37 and above  Income against clicked on ad<jupyter_code>incomePlot = qplot(clicked_on_ad, income, data=df, geom=c("boxplot", "jitter"), 
                   main="Income versus clicked_on_ad",xlab="clicked on ad", ylab="Income")
incomePlot<jupyter_output><empty_output><jupyter_text>Viewers earning an 40,000 to 80,000 were more likely to click on the ads as compared to those less likely to click on the ads, those who earn below 40,000<jupyter_code>usagePlot = qplot(clicked_on_ad, internet_usage, data=df, geom=c("boxplot", "jitter"), 
                   main="Internet Usage versus clicked_on_ad",xlab="clicked on ad", ylab="Internet Usage")
usagePlot<jupyter_output><empty_output><jupyter_text>The viewers more likely to view the ad have more internet usage of about 150 - 250 .  Gender against clicked on ad<jupyter_code>plot(df$clicked_on_ad,df$gender, ylab = "Gender", xlab = "Clicked on Ad")<jupyter_output><empty_output><jupyter_text>Generally more females clicked on ad as compared to males  Question OneWhich months had more internet usage?<jupyter_code># plot the data using ggplot2
ggplot(data=df_date, aes(x =month, y = internet_usage)) +
      geom_point(color = "darkorchid4") +
      labs(title = "Internet usage of viewers per month")<jupyter_output><empty_output><jupyter_text>The months with most internet usage include the first and the seventh month  Question TwoWhich gender is more likely to click on the ads depending on their internet usage?<jupyter_code># Stacked barplot with multiple groups
ggplot(data=df, aes(x=clicked_on_ad, y=internet_usage, fill=gender)) +
  geom_bar(stat="identity",position=position_dodge())<jupyter_output><empty_output><jupyter_text>More females generally have more internet usage as compared to men, More males are likely to click on the ad as compared to females.  Question Three:In which month have the least click on ads based on the time spent on site?<jupyter_code># Stacked barplot with multiple groups
ggplot(data=df, aes(x=month, y=time_on_site, fill=clicked_on_ad)) +
  geom_bar(stat="identity",position=position_dodge())<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11700634814567323, 'fraction_numerical': 0.026662211827597728, 'mean_word_length': 5.515454941227688, 'pattern_counts': {'":': 0, '<': 144, '<?xml version=': 0, '>': 123, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '25', 'dir': 'jupyter-structured-clean-dedup', 'id': '373615', 'license': 'MIT', 'n_tokens_mistral': 5485, 'n_tokens_neox': 4933, 'n_words': 1515, 'path': 'Data_cleaning_with_R.ipynb', 'repo_name': 'kinyalaura/Detailed-Exploratory-Data-Analysis-using-R', 'size': '14969'}