<commit_before>---
layout: post
title:  "SAT Scores"
date:   2017-03-26 21:55:09 -0400
categories: how-to
---
 <script type='text/javascript'>                    var divElement = document.getElementById('viz1497373854939');                    var vizElement = divElement.getElementsByTagName('object')[0];                    vizElement.style.width='100%';vizElement.style.height=(divElement.offsetWidth*0.75)+'px';                    var scriptElement = document.createElement('script');                    scriptElement.src = 'https://public.tableau.com/javascripts/api/viz_v1.js';                    vizElement.parentNode.insertBefore(scriptElement, vizElement);                </script>
 
For a project, I was given SAT score data from eligible students--those who were in high school and took the SAT in 2001. We were told simply to see what insights we could draw. After my initial scan of the data, what immediately stood out to me was the number of low participation scores--percentage of high school students who took the SAT--specifically in the Midwest. 

This brought to mind a conversation I had with my freshman year college roommate, because obviously freshman in their first month of college think it's still relevant to talk about standardized tests. My roommate, from Chicago, mentioned that all of her classmates were taking the ACT, but none of the schools she wanted to attend required the ACT. She was primarily looking at East Coast schools. I then thought about my own classmates on the East Coast who would take and retake the SATs, but pay no mind to the ACTs. It seemed like a pretty clear regional divide.

To make sure I wasn't just zoning in on a hunch but instead looking at the dataset as a whole, I mapped out the Verbal SAT scores by state:

First, I mapped the Verbal SAT scores by state:

<div class='tableauPlaceholder' id='viz1497373854939' style='position: relative'><noscript><a href='#'><img alt='2001 Average Verbal SAT Scores by State ' src='https:&#47;&#47;public.tableau.com&#47;static&#47;images&#47;20&#47;2001AverageVerbalSATScoresbyState&#47;Sheet4&#47;1_rss.png' style='border: none' /></a></noscript><object class='tableauViz'  style='display:none;'><param name='host_url' value='https%3A%2F%2Fpublic.tableau.com%2F' /> <param name='site_root' value='' /><param name='name' value='2001AverageVerbalSATScoresbyState&#47;Sheet4' /><param name='tabs' value='no' /><param name='toolbar' value='yes' /><param name='static_image' value='https:&#47;&#47;public.tableau.com&#47;static&#47;images&#47;20&#47;2001AverageVerbalSATScoresbyState&#47;Sheet4&#47;1.png' /> <param name='animate_transition' value='yes' /><param name='display_static_image' value='yes' /><param name='display_spinner' value='yes' /><param name='display_overlay' value='yes' /><param name='display_count' value='yes' /></object></div>               

Pretty interesting that the highest Verbal scores are concentrated in the Midwest. Because I'm from the Northeast, I'm totally biased and think all other regions are inferior. Seeing that the Northeast performed the worst out of the regions was quite disheartening. Perhaps this odd regional loyalty led me to consider the participation rates. But before I did that, I checked out the Math scores by state:

<div class='tableauPlaceholder' id='viz1489527750572' style='position: relative'><noscript><a href='#'><img alt='2001 Average Math SAT Scores by State ' src='https:&#47;&#47;public.tableau.com&#47;static&#47;images&#47;Av&#47;AverageMathSATScoresbyState&#47;Sheet2&#47;1_rss.png' style='border: none' /></a></noscript><object class='tableauViz'  style='display:none;'><param name='host_url' value='https%3A%2F%2Fpublic.tableau.com%2F' /> <param name='path' value='views&#47;AverageMathSATScoresbyState&#47;Sheet2?:embed=y&amp;:display_count=y' /> <param name='toolbar' value='yes' /><param name='static_image' value='https:&#47;&#47;public.tableau.com&#47;static&#47;images&#47;Av&#47;AverageMathSATScoresbyState&#47;Sheet2&#47;1.png' /> <param name='animate_transition' value='yes' /><param name='display_static_image' value='yes' /><param name='display_spinner' value='yes' /><param name='display_overlay' value='yes' /><param name='display_count' value='yes' /></object></div>

Turns out the Northeast performed worse than the Midwest on the math section as well. As I mentioned above, the participation rate was far more interesting:

<div class='tableauPlaceholder' id='viz1490032944746' style='position: relative'><noscript><a href='#'><img alt='2001 SAT Participation Rate by State ' src='https:&#47;&#47;public.tableau.com&#47;static&#47;images&#47;20&#47;2001SATParticipationRatebyState_0&#47;Sheet3&#47;1_rss.png' style='border: none' /></a></noscript><object class='tableauViz'  style='display:none;'><param name='host_url' value='https%3A%2F%2Fpublic.tableau.com%2F' /> <param name='site_root' value='' /><param name='name' value='2001SATParticipationRatebyState_0&#47;Sheet3' /><param name='tabs' value='no' /><param name='toolbar' value='yes' /><param name='static_image' value='https:&#47;&#47;public.tableau.com&#47;static&#47;images&#47;20&#47;2001SATParticipationRatebyState_0&#47;Sheet3&#47;1.png' /> <param name='animate_transition' value='yes' /><param name='display_static_image' value='yes' /><param name='display_spinner' value='yes' /><param name='display_overlay' value='yes' /><param name='display_count' value='yes' /></object></div>

Alright, so more Northeastern high school students take the SAT. I can see the light. The Midwest pretty clearly is not concerned with the SATs, in the same way that the East Coast doesn't bother with the ACTs. Also, this smaller sample helps me reason with the higher averages of the Midwest scores, as smaller samples are much harder to find a reliable mean for. My overachieving peers took the ACT to affirm their aptitude, as maybe Midwestern students may do so with the SAT.

I scoured the internet to find comparable ACT data. While the samples aren't identical--SAT dataset is highschoolers who took the exam in 2001; ACT dataset is the percentage of highschool *graduates* who took their exam in 2001--I can still use what available data I have to better understand the Midwest's test taking preferences.<commit_msg>Create 2017-03-19-SAT-scores.md<commit_after>---
layout: post
title:  "SAT Scores"
date:   2017-03-26 21:55:09 -0400
categories: how-to
---
 
For a project, I was given SAT score data from eligible students--those who were in high school and took the SAT in 2001. We were told simply to see what insights we could draw. After my initial scan of the data, what immediately stood out to me was the number of low participation scores--percentage of high school students who took the SAT--specifically in the Midwest. 

This brought to mind a conversation I had with my freshman year college roommate, because obviously freshman in their first month of college think it's still relevant to talk about standardized tests. My roommate, from Chicago, mentioned that all of her classmates were taking the ACT, but none of the schools she wanted to attend required the ACT. She was primarily looking at East Coast schools. I then thought about my own classmates on the East Coast who would take and retake the SATs, but pay no mind to the ACTs. It seemed like a pretty clear regional divide.

To make sure I wasn't just zoning in on a hunch but instead looking at the dataset as a whole, I mapped out the Verbal SAT scores by state:

First, I mapped the Verbal SAT scores by state:

<div class='tableauPlaceholder' id='viz1497373854939' style='position: relative'><noscript><a href='#'><img alt='2001 Average Verbal SAT Scores by State ' src='https:&#47;&#47;public.tableau.com&#47;static&#47;images&#47;20&#47;2001AverageVerbalSATScoresbyState&#47;Sheet4&#47;1_rss.png' style='border: none' /></a></noscript><object class='tableauViz'  style='display:none;'><param name='host_url' value='https%3A%2F%2Fpublic.tableau.com%2F' /> <param name='site_root' value='' /><param name='name' value='2001AverageVerbalSATScoresbyState&#47;Sheet4' /><param name='tabs' value='no' /><param name='toolbar' value='yes' /><param name='static_image' value='https:&#47;&#47;public.tableau.com&#47;static&#47;images&#47;20&#47;2001AverageVerbalSATScoresbyState&#47;Sheet4&#47;1.png' /> <param name='animate_transition' value='yes' /><param name='display_static_image' value='yes' /><param name='display_spinner' value='yes' /><param name='display_overlay' value='yes' /><param name='display_count' value='yes' /></object></div>                <script type='text/javascript'>                    var divElement = document.getElementById('viz1497373854939');                    var vizElement = divElement.getElementsByTagName('object')[0];                    vizElement.style.width='100%';vizElement.style.height=(divElement.offsetWidth*0.75)+'px';                    var scriptElement = document.createElement('script');                    scriptElement.src = 'https://public.tableau.com/javascripts/api/viz_v1.js';                    vizElement.parentNode.insertBefore(scriptElement, vizElement);                </script>            

Pretty interesting that the highest Verbal scores are concentrated in the Midwest. Because I'm from the Northeast, I'm totally biased and think all other regions are inferior. Seeing that the Northeast performed the worst out of the regions was quite disheartening. Perhaps this odd regional loyalty led me to consider the participation rates. But before I did that, I checked out the Math scores by state:

<div class='tableauPlaceholder' id='viz1489527750572' style='position: relative'><noscript><a href='#'><img alt='2001 Average Math SAT Scores by State ' src='https:&#47;&#47;public.tableau.com&#47;static&#47;images&#47;Av&#47;AverageMathSATScoresbyState&#47;Sheet2&#47;1_rss.png' style='border: none' /></a></noscript><object class='tableauViz'  style='display:none;'><param name='host_url' value='https%3A%2F%2Fpublic.tableau.com%2F' /> <param name='path' value='views&#47;AverageMathSATScoresbyState&#47;Sheet2?:embed=y&amp;:display_count=y' /> <param name='toolbar' value='yes' /><param name='static_image' value='https:&#47;&#47;public.tableau.com&#47;static&#47;images&#47;Av&#47;AverageMathSATScoresbyState&#47;Sheet2&#47;1.png' /> <param name='animate_transition' value='yes' /><param name='display_static_image' value='yes' /><param name='display_spinner' value='yes' /><param name='display_overlay' value='yes' /><param name='display_count' value='yes' /></object></div>

Turns out the Northeast performed worse than the Midwest on the math section as well. As I mentioned above, the participation rate was far more interesting:

<div class='tableauPlaceholder' id='viz1490032944746' style='position: relative'><noscript><a href='#'><img alt='2001 SAT Participation Rate by State ' src='https:&#47;&#47;public.tableau.com&#47;static&#47;images&#47;20&#47;2001SATParticipationRatebyState_0&#47;Sheet3&#47;1_rss.png' style='border: none' /></a></noscript><object class='tableauViz'  style='display:none;'><param name='host_url' value='https%3A%2F%2Fpublic.tableau.com%2F' /> <param name='site_root' value='' /><param name='name' value='2001SATParticipationRatebyState_0&#47;Sheet3' /><param name='tabs' value='no' /><param name='toolbar' value='yes' /><param name='static_image' value='https:&#47;&#47;public.tableau.com&#47;static&#47;images&#47;20&#47;2001SATParticipationRatebyState_0&#47;Sheet3&#47;1.png' /> <param name='animate_transition' value='yes' /><param name='display_static_image' value='yes' /><param name='display_spinner' value='yes' /><param name='display_overlay' value='yes' /><param name='display_count' value='yes' /></object></div>

Alright, so more Northeastern high school students take the SAT. I can see the light. The Midwest pretty clearly is not concerned with the SATs, in the same way that the East Coast doesn't bother with the ACTs. Also, this smaller sample helps me reason with the higher averages of the Midwest scores, as smaller samples are much harder to find a reliable mean for. My overachieving peers took the ACT to affirm their aptitude, as maybe Midwestern students may do so with the SAT.

I scoured the internet to find comparable ACT data. While the samples aren't identical--SAT dataset is highschoolers who took the exam in 2001; ACT dataset is the percentage of highschool *graduates* who took their exam in 2001--I can still use what available data I have to better understand the Midwest's test taking preferences.
{'fraction_non_alphanumeric': 0.1281969564178153, 'fraction_numerical': 0.04334315990757708, 'mean_word_length': 5.755651237890205, 'pattern_counts': {'":': 0, '<': 123, '<?xml version=': 0, '>': 123, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6414618', 'n_tokens_mistral': 4023, 'n_tokens_neox': 3503, 'n_words': 1220}