<commit_before>---
title: "Notes on Datacamp's kaggle tutorial"
excerpt_separator: "<!--more-->"
categories:
  - Data Science
tags:
  - R
---
Kaggle has a tutorial competition on the survival of the passengers of the Titanic. Datacamp has a handy tutorial on using R to tackle the problem. Here are my notes working through the tutorial
<!--more-->

I am posting this tutorial as I learn R. I will respond to feedback for errata in the comments.
{: .notice}

I will not address any issues that we have covered in the getting started section. Follow along at [Datacamp's open course on machine learning and the sinking of the Titanic](https://www.datacamp.com/community/open-courses/kaggle-tutorial-on-machine-learing-the-sinking-of-the-titanic)


## How it works
Basic information on navigating the tutorial. Basically instructions are on the left, the script file you submit is at the top right and the console where you can experiment is at the bottom right. Using `?` in front of a topic brings up help in a new tab at the top right as a separate tab page to the script file.

## Set sail
### read.csv()
`read.csv()` is essentially just a call to `read.table()` with some defaults provided to make it better suited for comma separated value files. `read.table` and its variants `read.csv()`, `read.csv2()`, `read.delim()` and `read.delim2()` all are used for reading files and producing data frames. In case you are wondering, the variants ending in 2 are for files that have ',' as a decimal separator. As a developer this horrifies me, it would have been better simply to override the default `dec` parameter in `read.csv` with `dec = ","`.

### url()
The input to `read.table()` and by extension `read.csv()` is a file that can be provided simply by specifying a file name or via a "connection", which I understand to be analogous or perhaps equivalent to a text stream. In this sample the connection used is by using the `url()` function. The connections topic has more on this topic but the `read.table()` function takes care of opening, reading and closing the connector, we simply hav eto construct it and pass it as a parameter

`read.csv()` seems to operate fine when passing a url directly as a file location like this `read.csv(url_to_a_csv)`, I assume there is some parser magic happening that recognizes that the path provided is a URL. But to be explicit we can always call it like this: `read.csv(url(url_to_a_csv))` 

### print()
I have not shown you print, mostly because just calling a value will print its values automatically in the console. For your information `print()` is a more explict way to oputput data. As far as this tutorial at Datacamp, it doesn't seem to matter if you just list the variables in the script or actually call print on them to pass the exercise

## Understanding your data
I've already commented on the `str()` or structure function. In this case where we have a larger dataset you can now see how much more useful it is that with my simple three row dataset. To recap : Database table analog: observations are like records, variables are fields. Below the observation you can see the dataframe transposed so we can see each variable's name, type and a sample of some of its data. You'll also see that strings were converted to factors automatically

## Rose vs Jack, or Female vs Male
### table()
The `table()` function is used for cross tabulation. In this case a single variable was passed and the count of each instance is reported. If we had passed more variables we would have counts based on combinations.<commit_msg>Title tags<commit_after>---
title: "Notes on Datacamp's Kaggle R tutorial"
excerpt_separator: "<!--more-->"
categories:
  - Data Science
tags:
  - R
  - Kaggle
  - Datacamp
---
Kaggle has a tutorial competition on the survival of the passengers of the Titanic. Datacamp has a handy tutorial on using R to tackle the problem. Here are my notes working through the tutorial
<!--more-->

I am posting this tutorial as I learn R. I will respond to feedback for errata in the comments.
{: .notice}

I will not address any issues that we have covered in the getting started section. Follow along at [Datacamp's open course on machine learning and the sinking of the Titanic](https://www.datacamp.com/community/open-courses/kaggle-tutorial-on-machine-learing-the-sinking-of-the-titanic)


## How it works
Basic information on navigating the tutorial. Basically instructions are on the left, the script file you submit is at the top right and the console where you can experiment is at the bottom right. Using `?` in front of a topic brings up help in a new tab at the top right as a separate tab page to the script file.

## Set sail
### read.csv()
`read.csv()` is essentially just a call to `read.table()` with some defaults provided to make it better suited for comma separated value files. `read.table` and its variants `read.csv()`, `read.csv2()`, `read.delim()` and `read.delim2()` all are used for reading files and producing data frames. In case you are wondering, the variants ending in 2 are for files that have ',' as a decimal separator. As a developer this horrifies me, it would have been better simply to override the default `dec` parameter in `read.csv` with `dec = ","`.

### url()
The input to `read.table()` and by extension `read.csv()` is a file that can be provided simply by specifying a file name or via a "connection", which I understand to be analogous or perhaps equivalent to a text stream. In this sample the connection used is by using the `url()` function. The connections topic has more on this topic but the `read.table()` function takes care of opening, reading and closing the connector, we simply hav eto construct it and pass it as a parameter

`read.csv()` seems to operate fine when passing a url directly as a file location like this `read.csv(url_to_a_csv)`, I assume there is some parser magic happening that recognizes that the path provided is a URL. But to be explicit we can always call it like this: `read.csv(url(url_to_a_csv))` 

### print()
I have not shown you print, mostly because just calling a value will print its values automatically in the console. For your information `print()` is a more explict way to oputput data. As far as this tutorial at Datacamp, it doesn't seem to matter if you just list the variables in the script or actually call print on them to pass the exercise

## Understanding your data
I've already commented on the `str()` or structure function. In this case where we have a larger dataset you can now see how much more useful it is that with my simple three row dataset. To recap : Database table analog: observations are like records, variables are fields. Below the observation you can see the dataframe transposed so we can see each variable's name, type and a sample of some of its data. You'll also see that strings were converted to factors automatically

## Rose vs Jack, or Female vs Male
### table()
The `table()` function is used for cross tabulation. In this case a single variable was passed and the count of each instance is reported. If we had passed more variables we would have counts based on combinations.
{'fraction_non_alphanumeric': 0.06582066657369963, 'fraction_numerical': 0.000836703388648724, 'mean_word_length': 4.261922230374175, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 7, 'https://': 2, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2757365', 'n_tokens_mistral': 1834, 'n_tokens_neox': 1745, 'n_words': 1145}