<jupyter_start><jupyter_text>ScatterplotsBy <NAME>, <NAME>, <NAME> What is a scatterplot?A scatterplot is a visualization where 2 quantitative variables of an observation are plotted against each other on the x and y axis. What is a scatterplot used for?Scatterplots are used to visualize the correlation between 2 quantitative variables, i.e. does a certain value of one variable increase the likelihood of a higher or lower value of the other variable?Some good examples of where a scatterplot could be useful:- You survey students for their GPA and SAT score, you can plot each individual's GPA and SAT score in a scatterplot to see if there is the strength of the relationship between GPA and SAT score.- You run an experiment to measure the rate constant of a reaction at different temperatures. You can plot the natural log of the rate constant over inverse temperature to see if the reaction follows the Arrhenius equation. ExerciseConsider the [Business Dynamics Dataset](https://corgis-edu.github.io/corgis/csv/business_dynamics/) from the CORGIS Project. Come up with an example of two quantitative variables that could be plotted against each other in a scatterplot. Number of Firms and Net Job Creation  What are some mistakes people make when using scatterplots?- *Using the correct values*: The two values of each point should be associated with each other! The two values you're plotting should connect to the same observation If you have a list of GPAs and a list of SAT scores but don't know which GPA is attached to the same person as which score, you can't make a scatterplot.- *Overplotting*: Don't overplot the graph to the point where it's just a massive blob with no distinguishable points.- *Using enough points*: With too few points, it could look like something has a stronger or weaker relationship than it actually does. As in most graphs, more data is better with the correct spacing.- *Non-linear relationships*: Depending on the data, two variables can be related in shape other than a straight line. Especially when calculating the numeric value of correlation, you may need to transform the data with a function, like logarithm or exponentiating. A perfect parabola can have a 0 correlation, which would indicate no relationship.- *Correlation is not causation*: As we saw with the pirate example in class, make sure you don't make assumptions about causation. When discussing scatterplots, be sure to always say that the two values are correlated or related, rather than that one causes the other, unless you have an experiment to prove it.  What are some important variations in scatterplots?- *Three variables*: More than two variables can be represented by changing the size of each marker to respresent a change in a quantitative variable or something like color for qualitative variables. An additional axis can also be added, but it's pretty ugly and bad.- *Trendlines*: Adding trendlines helps to better represent the relationship between two variables.- *Correlation*: The value of correlation, `r`, is often calculated and displayed alongside scatterplots. It is a value between -1 and 1, where the greater the absolute value of `r`, the stronger the linear relationship between the two variables. Correlation can be calculated using the `corr()` method in Pandas.  What are some alternatives to scatterplots?- *2D Density Plot*: As shown on [this website](https://www.chrisstucchio.com/blog/2012/dont_use_scatterplots.html), plotting the density of the points instead of plotting the actual points cna provide a much easier to read graph if there is an extremely large amount of data.Other than this, scatterplots are relatively unique in the way that they reflect on variable's relation to another.  Okay, but can we make Scatter Plots in matplotlib?Can use the plt.scatter() function, passing a pandas series in the x and y. Need to seperately label the x and y axis. business_dynamics.csv is from the [CORGIS project site](https://corgis-edu.github.io/corgis/csv/business_dynamics/)Example code is given below:<jupyter_code># matplotlib example
# Example altered from https://stackabuse.com/matplotlib-scatterplot-tutorial-and-examples/
import matplotlib.pyplot as plt
import pandas as pd

# Reading data from CORGIS website
df = pd.read_csv('https://corgis-edu.github.io/corgis/datasets/csv/business_dynamics/business_dynamics.csv')
df.head()

# Data is shown below in dataframe
# Increases the plot size in the Jupyter notebook; this part is not necessary
fig, ax = plt.subplots(figsize=(10, 6))

#Grabbing the first decade of the 2000s from the data set
df_specified_years = df[(df.Year >= 2000) & (df.Year <= 2009)]

# Plots the Job Creation Rate on the x axis against the Business Creation Rate for each state for the year 2000
plt.scatter(x = df_specified_years["Data.Calculated.Net Job Creation Rate"], y = df_specified_years["Data.Establishments.Entered Rate"])

# Applies x and y axis labels and a title
plt.xlabel("New business creation rate")
plt.ylabel("New job creation rate")
plt.title("New Job Creation Rate vs. Business Creation Rate in Each US State, Each Year, 2000-2009")

# Displays the plot we've been editing
plt.show()<jupyter_output><empty_output><jupyter_text>Okay, but what about Seaborn?You can also use the seaborn.scatterplot() function, passing a dataframe as the data and then selecting the x and y axis through passing the column.Example code is given below:<jupyter_code># seaborn example
# Example altered from https://stackabuse.com/matplotlib-scatterplot-tutorial-and-examples/
import matplotlib.pyplot as plt
import seaborn as sns
import pandas as pd

# Reading data from CORGIS
df = pd.read_csv('https://corgis-edu.github.io/corgis/datasets/csv/business_dynamics/business_dynamics.csv')

# Grabbing only one decade of data
df_specified_years = df[(df.Year >= 2000) & (df.Year <= 2009)]

# Create scatterplot - note: labels come from dataframe
scatter = sns.scatterplot(data = df_specified_years, x = "Data.Calculated.Net Job Creation Rate", y = "Data.Establishments.Entered Rate")
# Set scatterplot title
scatter = scatter.set_title("New Job Creation Rate vs. Business Creation Rate in Each US State, Each Year, 2000-2009")
scatter
plt.show()<jupyter_output><empty_output><jupyter_text>ExerciseUse either Matplotlib or Seaborn to plot the two variables you selected above!<jupyter_code># Put code here:
import matplotlib.pyplot as plt
import pandas as pd
df = pd.read_csv('https://corgis-edu.github.io/corgis/datasets/csv/business_dynamics/business_dynamics.csv')
df_specified_years = df[(df.Year >= 2000) & (df.Year <= 2009)]

plt.scatter(x = df_specified_years["Data.Number of Firms"], y = df_specified_years["Data.Calculated.Net Job Creation"])

plt.xlabel("Number of Firms")
plt.ylabel("New job creation rate")
plt.title("Number of Firms vs. New Job Creation Rate, Each Year, 2000-2009")

plt.show()<jupyter_output><empty_output><jupyter_text>What about adding a trendline?Can use the regression plot function which will add a trendline if passed the same information as above in seaborn but using the lmplot function instead. Must use other statistics packages to find the goodness of fit, which is not covered here but this [stack overflow article](https://stackoverflow.com/questions/25579227/seaborn-implot-with-equation-and-r2-text) covers it.Note that this function can be used to fit several lines of best fit, with linear as the default, more info is [here](https://seaborn.pydata.org/tutorial/regression.html)<jupyter_code>sns.lmplot(data = df_specified_years, y = "Data.Calculated.Net Job Creation Rate", x = "Data.Establishments.Entered Rate")
plt.show()<jupyter_output><empty_output><jupyter_text>ExerciseTry adding the trendline!<jupyter_code># Put code here:
plt.lmplot(data = df_specified_years, y = "Data.Calculated.Net Job Creation Rate", x = "Data.Establishments.Entered Rate")<jupyter_output><empty_output><jupyter_text>Checking your understanding1. What common problem in scatterplot design is present in the following example that uses data from all years?<jupyter_code>scatter = sns.scatterplot(data = df, x = "Data.Calculated.Net Job Creation Rate", y = "Data.Establishments.Entered Rate")
scatter = scatter.set_title("New Job Creation Rate vs. Business Creation Rate in Each US State, Each Year, 2000-2009")
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07104413347685684, 'fraction_numerical': 0.010405453893074991, 'mean_word_length': 5.268365817091454, 'pattern_counts': {'":': 0, '<': 31, '<?xml version=': 0, '>': 31, 'https://': 10, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '234994', 'license': 'MIT', 'n_tokens_mistral': 2423, 'n_tokens_neox': 2211, 'n_words': 1061, 'path': 'JigsawLessons/.ipynb_checkpoints/scatter_plots-checkpoint.ipynb', 'repo_name': 'ithakker/CISC367-Projects', 'size': '8385'}