<jupyter_start><jupyter_text>Tidy data<jupyter_code># Ignore this (only for notebooks):
library(repr)
options(repr.plot.width=15, repr.plot.height=10)
library(tidyverse)
library(fivethirtyeight)<jupyter_output>── [1mAttaching packages[22m ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── tidyverse 1.3.0 ──

[32m✔[39m [34mggplot2[39m 3.3.3     [32m✔[39m [34mpurrr  [39m 0.3.4
[32m✔[39m [34mtibble [39m 3.1.0     [32m✔[39m [34mdplyr  [39m 1.0.4
[32m✔[39m [34mtidyr  [39m 1.1.2     [32m✔[39m [34mstringr[39m 1.4.0
[32m✔[39m [34mreadr  [39m 1.4.0     [32m✔[39m [34mforcats[39m 0.5.1

── [1mConflicts[22m ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── tidyverse_conflicts() ──
[31m✖[39m [34mdplyr[39m::[32mfilter()[39m masks [34mstats[39m::filter()
[31m✖[39m [34mdplyr[39m::[32mlag()[39m    masks [34mstats[39m::lag()

Some larger datasets need to be installe[...]<jupyter_text>3. Now it's your turn **Task A**Inspect the `police_locals` dataset. Here is the article attached to it: https://fivethirtyeight.com/features/most-police-dont-live-in-the-cities-they-serve/.1. Create a new dataset, called `police_locals_new`, made of the following columns: `city`, `force_size`, `ethnic_group`, `perc_locals`. You should create the `ethnic_group` column using a pivot function, as shown in this wpa or in wpa_4. The values in this column should be `all`, `white`, `non_white`, `black`, `hispanic`, `asian`. The `perc_locals` column should contain the percentage of officers that live in the town where they work, corresponding to their ethnic group. Rearrange based on the `ethnic_group` and inspect it by printing the first 10 lines.2. Make a boxplot, with `ethnic_group` on the x-axis and `perc_locals` on the y-axis. `ethnic_group` should be ordered from the lowest `perc_locals` to the highest. Put appropriate labels.**Task B**Inspect the `unisex_names` dataset. Here is the article attached to it: https://fivethirtyeight.com/features/there-are-922-unisex-names-in-america-is-yours-one-of-them/.1. Create a new dataset, called `unisex_names_new`, made of the following columns: `name`, `total`, `gap`, `gender`, `share`. The `gender` column should only contain the values "male" and "female". The `share` column should contain the percentages. 2. Multiply the `share` column by 100. Re-arrange so that the first rows contain the names with the highest `total`. Print the first 10 rows of the newly created dataset. Create now a new dataset, called `unisex_names_common`, with the names in `unisex_names_new` that have a total higher than 50000.2. Using `unisex_names_common`, make a barplot that has `share` on the y-axis, `name` on the x-axis, and with each bar split vertically by color based on the `gender`.**Task C**Inspect the `tv_states` dataset. Here is the article attached to it: https://fivethirtyeight.com/features/the-media-really-started-paying-attention-to-puerto-rico-when-trump-did/.1. Create a new dataset, with a different name, that is the long version of `tv_states`. You should decide how to call the new columns, as well as which columns should be used for the transformation.2. With the newly created dataset, make a plot of your choosing to illustrate the information contained in this dataset. As an inspiration, you can have a look at what plot was done in the article above.  Task A<jupyter_code>head(police_locals)
police_locals_new = police_locals %>%
    pivot_longer(cols = all:asian,
                 names_to = "ethnic_group",
                 values_to = "perc_locals",
                 names_repair = "minimal") %>%
    arrange(ethnic_group)

head(police_locals_new, 30)
ggplot(data = police_locals_new, mapping = aes(x = reorder(ethnic_group, perc_locals), y = perc_locals)) +
    geom_boxplot() +
    labs(x = 'Ethnic group', y = 'Mean Percentage Locals') + 
    theme(axis.text.x = element_text(angle = 90))<jupyter_output>Warning message:
“Removed 111 rows containing non-finite values (stat_boxplot).”<jupyter_text>Task B<jupyter_code>head(unisex_names)
unisex_names_new = unisex_names %>%
    pivot_longer(cols = male_share:female_share,
                 names_to = "gender",
                 values_to = "share",
                 names_repair = "minimal") %>%
    separate(col = gender,
             into = c('gender', 'to_delete'),
             sep = '_',
             remove = TRUE) %>%
    select(-to_delete) %>%
    mutate(share = share*100) %>%
    arrange(desc(total))

dim(unisex_names_new)
head(unisex_names_new, 10)
unisex_names_common = unisex_names_new %>%
    filter(total > 50000)
ggplot(data = unisex_names_common, mapping = aes(x = name, y = share, fill = gender)) +
    geom_col() +
    labs(x = 'Name', y = 'Share', fill='Gender') + 
    theme(axis.text.x = element_text(angle = 90))<jupyter_output><empty_output><jupyter_text>Task C<jupyter_code>head(tv_states)
tv_states_new = pivot_longer(tv_states,
                             cols = florida:puerto_rico,
                             names_to = 'state',
                             values_to = 'share_of_sentences')

dim(tv_states_new)
head(tv_states_new, 10)
ggplot(data = tv_states_new, mapping = aes(x = date, y= share_of_sentences, fill = state)) + 
    geom_col() +
    labs(x = 'Date', fill = 'State', y= 'Share of sentences')
ggplot(tv_states_new, aes(x=date, y=share_of_sentences, group=state, color=state)) +  
    geom_line() + 
    labs(x = 'Date', y = 'News coverage (%)', color='Area') + 
    scale_color_manual(values=c("#54F708", "blue", "red")) + 
    theme(axis.text.x = element_text(angle = 90, size = 6))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.18898834656308056, 'fraction_numerical': 0.03158250295558183, 'mean_word_length': 4.503717472118959, 'pattern_counts': {'":': 0, '<': 15, '<?xml version=': 0, '>': 24, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '543976', 'license': 'MIT', 'n_tokens_mistral': 2300, 'n_tokens_neox': 1952, 'n_words': 533, 'path': 'notebooks/wpa6_answers.ipynb', 'repo_name': 'laurafontanesi/r-seminar', 'size': '5921'}