<jupyter_start><jupyter_text>Mapping from `datascience` to R This notebook serves as an introduction to basic R terminology, data structures and commands. The functions introduced will be analogous to those in Berkeley's `datascience` module, with examples provided for each.We will cover the following topics in this notebook:1. Basics of R    - Importing and Loading Packages    - Arithmetic and Logical Operators    - Assigning Variables2. Dataframes: Storing Tabular Data    - Creating a Dataframe    - Accessing Values in Dataframe    - Manipulating Data3. Visualizing Data    - Histograms    - Line and Scatter Plots    - Bar Charts  1. Basics of R R is a command line driven program. This means that the user can enter expressions, create variables and define functions and run them in the R console. In the Jupyter notebook interface, code chunks can be run as individual cells either by clicking on 'Run' in the toolbar above or using the shortcut keys `shift + enter`  1.1 Importing and Loading Packages In Python, we use the following syntax to install packages:```python!pip install datascience```And we load them using:```pythonimport numpy as npfrom datascience import Table```In R, we use `install.packages('package_name')` to install new packages from the CRAN repository. For a full list of available packages, refer to https://cran.r-project.org/web/packages/available_packages_by_name.htmlIt is not necessary to reinstall packages everytime we quit or reload an R session. Once we have a package installed, we can load it using `library('package_name')`.<jupyter_code># example: install a package
install.packages('ggplot2')
# example: loading a package
library('ggplot2')<jupyter_output><empty_output><jupyter_text>1.2 Arithmetic and Logical Operators  Arithmetic OperatorsHere are the basic arithmetic operations in Python:```pythonimport mathimport numpy as npprint(2 + 3)  add numbersprint(3**4)  powersprint(pow(3, 4))  powersprint(math.sqrt(4**4))  functionsprint(21 % 5)  21 mod 5print(math.log(10))  take logprint(math.exp(2))  exponentialprint(np.abs(-2))  absolute valueprint(2*math.pi)  mathematical constant scientific notationprint(5000000000 * 1000)print(5e9 * 1e3)```In Python, we need to import `numpy` and `math` for certain mathematical operations. In R, however, these capabilities are built-in so no imports are required.Running the following cells will demostrate some basic operations performed in R.<jupyter_code># adding two numbers
2 + 3
# raising to a power
3 ^ 4
# square roots
sqrt(4 ^ 4)
# 21 mod 5
21 %% 5 
# taking the log
log(10)
# exponential
exp(2) 
# using mathematical constants
2 * pi 
# absolute value
abs(-2)
# scientific notation
5e9 * 1e3<jupyter_output><empty_output><jupyter_text>Logical OperatorsNow, recall the logical operations in Python:```pythonprint((1 > 0) and (3 <= 5))print((1  5))print((3 == 9/3) or (2 < 1) )print(not(2 != 4/3))```In R, the logical operators are , >=, == for exact equality and != for inequality. `and`, `or`, `not` is replaced by `&`, `|`, `!`. The boolean values True/False in Python correspond to TRUE/FALSE in R (Notice the case difference). Run the cells below to see how logical operators work in R.<jupyter_code>(1 > 0) & (3 <= 5)
(1 < 0) | (3 > 5)
(3 == 9/3) | (2 < 1)
!(2 != 4/3)<jupyter_output><empty_output><jupyter_text>1.3 Assigning Variables **Assignment**In R, the assignment operator is `<-`. In most (not all) contexts, the `=` operator can be used as an alternative. It is recommended to use `<-` as standard usage to avoid mistakes.Variables names in R are case sensitive, which means `A` and `a` are different symbols and would refer to different variables.<jupyter_code># run this cell
val <- 3
print(val) # same usage of print function as in Python 3

Val <- 7 # case-sensitive!
print(Val)<jupyter_output>[1] 3
[1] 7<jupyter_text>**Vectors**Vectors in R are analogous to lists in Python. The syntax for vectors is of the form `c(a, b, c)` (c means we are declaring a vector datatype).```python A comparison: create a list of numbers in pythona = [0.125, 4.75, -1.3]a1 = np.array([0.125, 4.75, -1.3])```<jupyter_code># run this cell
a <- c(1, 2, 3)
a
b <- c(4, 5, 6)
b
# combine two vectors
ab <- c(a, b)
ab<jupyter_output><empty_output><jupyter_text>**Creating Sequences**In Python, we used `np.arange` to create a numpy array with a start, end and a step value as follows:```pythona = np.arange(4, 9, 1)  creates [4 5 6 7 8]```In R, we can use the `seq` function to do the same. The end element is inclusive in `seq`, unlike `np.arange`.<jupyter_code># run this cell
seq1 <- seq(from=4, to=9, by=1)
seq1 # Notice the output difference with np.arange<jupyter_output><empty_output><jupyter_text>There are more parameters available for the `seq` function. To pull up more information about an R function, we can use either `?seq` or `help(seq)`  Another important difference is that in R, indexing starts at 1, unlike 0 in Python.<jupyter_code>seq1[1] # extracting element at first index of vector<jupyter_output><empty_output><jupyter_text>2. Dataframes: Storing Tabular Data In Python's `datascience` module, we used `Table` to build our dataframes and used commands such as `select()`, `where()`, `group()`, `column()` etc. In this section, we will go over some basic commands to work with tabular data in R using Dataframes.  2.1 Creating a Dataframe In Python's `datascience` module that is used in Data 8, this is how we created tables from scratch by extending an empty table:```pythont = Table().with_columns([     'letter', ['a', 'b', 'c', 'z'],     'count',  [  9,   3,   3,   1],     'points', [  1,   2,   2,  10], ])``` In R, we can initialize a dataframe using `data.frame()`. For a full list of parameters and options, refer to [this guide](https://www.rdocumentation.org/packages/base/versions/3.5.0/topics/data.frame)When not specified, the function `data.frame` will coerce all character variables to factors. If you want to keep the strings as character variables, you need to specify `stringsAsFactors = FALSE`.<jupyter_code># example: creating a dataframe in R
t <- data.frame(letter = c('a', 'b', 'c', 'z'),
                count = c(9, 3, 3, 1),
                points = c(1, 2, 2, 10),
                stringsAsFactors = FALSE
               )
t<jupyter_output><empty_output><jupyter_text>More often, we will need to create a dataframe by importing data from a .csv file. In `datascience`, this is how we read data from a csv:```pythonTable.read_table('sample.csv')```In R, we can use `read.csv()` to read data from a csv file. For a full list of parameters, refer to [this guide](https://www.rdocumentation.org/packages/utils/versions/3.5.0/topics/read.table)<jupyter_code># example: reading baby.csv (Located in current working directory)
baby <- read.csv('baby.csv')
head(baby) # display first few rows of dataframe
# view data summary
summary(baby)
# example: Load csv from URL
sat <- read.csv('http://data8.org/textbook/notebooks/sat2014.csv')
head(sat)
# view information about dataframe
summary(baby) # view data summary
nrow(sat) # display no. of rows
dim(sat) # view dimensions (rows, cols)
colnames(sat) # view column names<jupyter_output><empty_output><jupyter_text>2.2 Accessing Values in Dataframe In `datascience`, we can use `column()` to access values in a particular column as follows:```pythonIn [10]: t.column('letter')Out[10]: array(['a', 'b', 'c', 'z'],       dtype='<U1')```   In R, to access values in a particular column, we can use the `$` sign or use the following syntax: `df[, colname]`<jupyter_code># accessing column values
t$letter
t[, 'letter'] # Can also use t[, 1] to access column at first index<jupyter_output><empty_output><jupyter_text>In Python, we can use `take()` to access a row:```pythonIn [34]: t.take[0:2]  the first and second rowsOut[34]: points | letter | count1      | a      | 92      | b      | 3```In R, we can use the following syntax to access row data: `df[rowname, ]`<jupyter_code># example: Access first row of dataframe
t[1:2, ]<jupyter_output><empty_output><jupyter_text>We can also access a specific value in the dataframe by specifiying the row and column as follows:<jupyter_code># extracting one value
t[1, 'letter']
# slicing the dataframe
t[1:3, 'count']<jupyter_output><empty_output><jupyter_text>2.3 Manipulating Data **Adding Columns**Adding a new column in `datascience` is done by the `with_columns()` function as follows:```pythonIn [23]: t.with_column('vowel', ['yes', 'no', 'no', 'no'])Out[23]: points | letter | count | vowel1      | a      | 9     | yes2      | b      | 3     | no2      | c      | 3     | no10     | z      | 1     | no```In R, we can use `data$newcolumn<-datavector` to add a new column to an existing dataframe.<jupyter_code># example: Adding a new column
t$vowel <- c('yes', 'no', 'no', 'no')
t<jupyter_output><empty_output><jupyter_text>We can also add an existing column to the dataframe, which might be manipulated and added as a new column.<jupyter_code># Example: Adding twice the count to the dataframe
t$double <- t$count * 2
t<jupyter_output><empty_output><jupyter_text>**Selecting Columns**In `datascience`, we used `select()` to subset the dataframe by selecting columns:```pythonIn [27]: t.select(['letter', 'points'])Out[27]: letter | pointsa      | 1b      | 2c      | 2z      | 10```In R, we can subset the dataframe by columns using the `df[, colnames(s)]` notation. We can pass in the column names as a vector or specify the indexes of the columns we want to use<jupyter_code># selecting columns 1 through 3
t[, 1:3]
# selecting columns by name
colstoselect <- c('letter', 'points')
t[, colstoselect]<jupyter_output><empty_output><jupyter_text>**Filtering Rows Conditionally**In `datascience`, we used `where()` to select rows according to a given condition:```pythonIn [35]: t.where('points', 2)  rows where points == 2Out[35]: points | letter | count2      | b      | 32      | c      | 3In [36]: t.where(t['count'] < 8)  rows where count < 8Out[36]: points | letter | count2      | b      | 32      | c      | 310     | z      | 1```In R, we use the following syntax to subset rows: `df[df$colname  value, ]` (Note the comma at the end!). The following examples will provide different ways of filtering the dataframe.<jupyter_code># rows where points = 2
t[t$points == 2, ]
# rows where count < 8
t[t['count'] <= 8, ] # we can also use df['colname'] instead of df$colname
# multiple conditions: points < 2 and vowel = 'yes'
t[t$points < 2 & t['vowel'] == 'yes', ]<jupyter_output><empty_output><jupyter_text>**Renaming Columns**In `datascience`, we used `relabeled()` to rename columns:```pythonIn [29]: t.relabeled('points', 'other name')Out[29]: other name | letter | count1          | a      | 92          | b      | 32          | c      | 310         | z      | 1```In R, we can use the following to rename columns: `colnames(df)[colnames(df) == oldcolname] <- newcolname`. What this essentially does is select the required column by filtering from `colnames(df)` and then assigns the new name to that column<jupyter_code># example: renaming double to twice
colnames(t)[colnames(t) == 'double'] <- 'twice'
t<jupyter_output><empty_output><jupyter_text>**Sorting Dataframe by Column**In `datascience` we used `sort()` to sort a Table according to the values in a column:```pythonIn [40]: t.sort('count')Out[40]: points | letter | count10     | z      | 12      | b      | 32      | c      | 31      | a      | 9```In R, we can use `order()` in the subset notation: `df[order(df$colname), ]`.<jupyter_code># sort by count
t[order(t$count), ]
# sort by descending order of count
t[order(-t$count), ]<jupyter_output><empty_output><jupyter_text>**Grouping and Aggregating**In `datascience`, we used `group()` and the `collect` argument to group a Table by a column and aggregrate values in another column:```pythonIn [42]: t.select(['count', 'points']).group('count', collect=sum)Out[42]: count | points sum1     | 103     | 49     | 1```In R, we can use `aggregate()` to group and aggregate values in our dataframe<jupyter_code># grouping by count and summing points
aggregate(points ~ count, t, sum)<jupyter_output><empty_output><jupyter_text>**Pivot Tables**In `datascience`, we used the `pivot()` function to build contingency tables:```pythonIn [44]: other_tableOut[44]: mar_status | empl_status     | countmarried    | Working as paid | 1married    | Working as paid | 1partner    | Not working     | 1partner    | Not working     | 1married    | Not working     | 1In [45]: other_table.pivot('mar_status', 'empl_status', 'count', collect=sum)Out[45]: empl_status     | married | partnerNot working     | 1       | 2Working as paid | 2       | 0```In R, one simple way of building a pivot table is to use the `table()` function<jupyter_code># read in couples.csv
couples <- read.csv('couples.csv', header = TRUE)
head(couples)<jupyter_output><empty_output><jupyter_text>`table` is a very basic function to create tabular results for categorical variables. We could use it to summarize a vector of categorical variables. For example, to count the number of females and males:<jupyter_code># count no. of females and males
table(couples$Gender)<jupyter_output><empty_output><jupyter_text>When `table` takes two vectors as input and among them at least one can be interpreted as a factor (categorical variable), we get the contingency table<jupyter_code># creating a pivot table
ratings_by_gender <- table(couples$Gender, couples$Relationship.Rating)
ratings_by_gender<jupyter_output><empty_output><jupyter_text>3. Visualizing Data In Python, we learned to plot data using histograms, line plots, scatter plots and histograms. In `datascience`, the corresponding functions were `hist()`, `plot()`, `scatter()` and `barh()`.In this section we will go through some examples of plots in R.  3.1 HistogramsIn R, we can simple use the `hist()` function to create a histogram. In this example, we will be using data from `baby.csv`. Recall that the baby data set contains data on a random sample of 1,174 mothers and their newborn babies. The column `birthwt` contains the birth weight of the baby, in ounces; `gest_days` is the number of gestational days, that is, the number of days the baby was in the womb. There is also data on maternal age, maternal height, maternal pregnancy weight, and whether or not the mother was a smoker.<jupyter_code>head(baby)
# plotting a histogram of birth weights
hist(baby$Birth.Weight, 
     col=c("darkblue"), # histogram color
     main = "Birth weights of babies",# plot titile
     xlab = "Birth weights (in ounces)", # x axis lable
     xlim = c(40, 180)) # x axis range<jupyter_output><empty_output><jupyter_text>3.2 Line and Scatter PlotsWe can use the `plot()` function in R to plot two vectors X and Y. The first argument is the X values and the second argument is the Y values. We can specify the `type` parameter and set it to `'l'` for line plots, `'p'` for scatter plots and `'b'` for both. For a full list of parameters, refer to [this guide](https://www.rdocumentation.org/packages/graphics/versions/3.5.0/topics/plot)<jupyter_code># line plot
plot(c(1, 2, 3, 4), c(10, 12, 23, 30), type='l', col=c("darkblue"))
# scatter plot
plot(c(1, 2, 3, 4), c(10, 12, 23, 30), type='p', col=c("darkblue"))<jupyter_output><empty_output><jupyter_text>3.3 Bar ChartsWe can plot categorical variables using a bar chart using the `barplot()` function in R. Recall the contingency table `ratings_by_gender`<jupyter_code>ratings_by_gender<jupyter_output><empty_output><jupyter_text>We will now create a bar chart using `barplot()`. For a full list of parameters, refer to [this guide](https://www.rdocumentation.org/packages/graphics/versions/3.5.0/topics/barplot)<jupyter_code># creating a bar chart
barplot(ratings_by_gender, 
        main="Rating by Gender", # Title of the plot
        xlab="Rating", # x axis label
        col=c("darkblue","yellow"), # specify color
        legend = rownames(ratings_by_gender), # legend
        beside=TRUE # plot bars for male and female side by side
       )<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11615346411720884, 'fraction_numerical': 0.022535386143531164, 'mean_word_length': 4.4923286737129215, 'pattern_counts': {'":': 0, '<': 135, '<?xml version=': 0, '>': 108, 'https://': 5, 'lorem ipsum': 0, 'www.': 4, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 1, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '26', 'dir': 'jupyter-structured-clean-dedup', 'id': '439154', 'license': 'MIT', 'n_tokens_mistral': 5317, 'n_tokens_neox': 4915, 'n_words': 1969, 'path': 'Learning R.ipynb', 'repo_name': 'ds-modules/R-Module', 'size': '16108'}