<jupyter_start><jupyter_text>Observations and Insights  Observations:1.) Comparing the number of total data points to the number of unique mouse id's per drug regimen, we infer that Capomulin and Ramicane are the 2 most successful drugs in keeping the test subjects alive2.) Comparing the 4 Drug Regimen, we can infer that Capomulin and Ramicane achieved far better results than Infubinol and Ceftamin3.) We can infer, based on the graph of Tumor size to Weight of Capomulin treated Mice, that Tumor Size is related to the Weight of the mouse<jupyter_code># Dependencies and Setup
import matplotlib.pyplot as plt
import pandas as pd
import scipy.stats as st
import numpy as np

# Study data files
mouse_metadata_path = "data/Mouse_metadata.csv"
study_results_path = "data/Study_results.csv"

# Read the mouse data and the study results
mouse_metadata = pd.read_csv(mouse_metadata_path)
study_results = pd.read_csv(study_results_path)

# Combine the data into a single dataset
study_metadata = pd.merge(mouse_metadata, study_results, on='Mouse ID', how="outer")
# Display the data table for preview

study_metadata
# Checking the number of mice.
num_mice = len(study_metadata['Mouse ID'].unique())
num_mice
# Getting the duplicate mice by ID number that shows up for Mouse ID and Timepoint. 
study_metadata[study_metadata.duplicated(["Mouse ID","Timepoint"])]
# Optional: Get all the data for the duplicate mouse ID. 
study_metadata[study_metadata.duplicated(["Mouse ID","Timepoint"], keep=False)]
# Create a clean DataFrame by dropping the duplicate mouse by its ID.
clean_df = study_metadata.drop_duplicates(subset = ["Mouse ID","Timepoint"], keep = 'first')
clean_df
# Checking the number of mice in the clean DataFrame.
len(clean_df["Mouse ID"].unique())<jupyter_output><empty_output><jupyter_text>Summary Statistics<jupyter_code># Generate a summary statistics table of mean, median, variance, standard deviation, and SEM of the tumor volume for each regimen
# This method is the most straighforward, creating multiple series and putting them all together at the end.

#Get List of Drugs
drug_regimen = clean_df["Drug Regimen"].unique()
#Create DataFrame to Populate
summary1_df = pd.DataFrame()

#Loop Through Drug List
for drug in drug_regimen:
    working_df = clean_df[clean_df["Drug Regimen"] == f"{drug}"]
    #Aggregate the Summary Stats and input into Summary DF
    stats = working_df["Tumor Volume (mm3)"].agg(['mean', 'median', 'var', 'std', 'sem'])
    summary1_df[f'{drug}'] = stats   

summary1_df
# Generate a summary statistics table of mean, median, variance, standard deviation, and SEM of the tumor volume for each regimen
# This method produces everything in a single groupby function

#Aggregate all Statistics at Once
summary2_df = clean_df[{"Drug Regimen","Tumor Volume (mm3)"}].groupby('Drug Regimen').agg(['mean', 'median', 'var', 'std', 'sem'])
summary2_df<jupyter_output><empty_output><jupyter_text>Bar and Pie Charts<jupyter_code># Generate a bar plot showing the total number of mice for each treatment throughout the course of the study using pandas. 

#Grouping Mice by Drug Regimen, Counting the Data Points
drug_mice_numb_df = clean_df[{"Mouse ID","Drug Regimen"}].groupby('Drug Regimen')
drug_mice_numb_series = drug_mice_numb_df["Mouse ID"].count()

#Plotting using Pandas and Setting Axes and Titles Using MatPlotLib
drug_mice_numb_series.plot.bar()
plt.ylabel('Number of Data Points')
plt.title('Data Points per Drug')
# Generate a bar plot showing the total number of mice for each treatment throughout the course of the study using pyplot.

#Creating the X axis and plotting it
drug_x_axis =drug_mice_numb_series.index
plt.bar(drug_x_axis,drug_mice_numb_series, width = 0.5)

#Set the X ticks vertically, setting axes labels, and plot title
plt.xticks(rotation = "vertical")
plt.xlabel('Drug Regimen')
plt.ylabel('Number of Data Points')
plt.title('Data Points per Drug')
plt.tight_layout
#Grouping Mice by Drug Regimen, Counting the Total Mice for Comparison
drug_mice_numb_df = clean_df[{"Mouse ID","Drug Regimen"}].groupby('Drug Regimen')
drug_mice_numb_unique = drug_mice_numb_df["Mouse ID"].nunique()

#Plotting using Pandas and Setting Axes and Titles Using MatPlotLib
drug_mice_numb_unique.plot.bar()
plt.ylabel('Number of Mice')
plt.title('Mice per Drug')
# Generate a pie plot showing the distribution of female versus male mice using pandas

#Grouping the Data by Mouse Sex
mice_sex = clean_df[{"Mouse ID","Sex"}].groupby('Sex')
mice_sex_pie = mice_sex['Sex'].count()

#Plotting the Data using Pandas
mice_sex_pie.plot.pie(y='Sex', title="Title", legend=False, autopct='%1.1f%%')

mice_sex_pie
# Generate a pie plot showing the distribution of female versus male mice using pyplot

#Plotting the Data using PyPlot
plt.pie(mice_sex_pie, labels = mice_sex_pie.index, autopct="%1.1f%%")
plt.title('Sex')<jupyter_output><empty_output><jupyter_text>Quartiles, Outliers and Boxplots<jupyter_code># Calculate the final tumor volume of each mouse across four of the treatment regimens:  
# Capomulin, Ramicane, Infubinol, and Ceftamin
# Start by getting the last (greatest) timepoint for each mouse
# Merge this group df with the original dataframe to get the tumor volume at the last timepoint

# Creating Empty DataFrame to Populate, Generating the List of Specific Drug Regimen
last_timepoint_df = pd.DataFrame()
specific_drug_list = ['Capomulin',"Ramicane",'Infubinol','Ceftamin']

# Filtering Data to only get Specific Drugs, Getting the Unique Mouse IDs
specific_regimen_df = clean_df[clean_df['Drug Regimen'].isin(specific_drug_list)]
mouse_list = specific_regimen_df['Mouse ID'].unique()

#Looping through all the Mice
for mouse in mouse_list:
    #Populating the DataFrame with the last Timepoint of every Mouse taking the Specific Drugs
    df = specific_regimen_df[specific_regimen_df['Mouse ID'] == mouse]
    volume_max = df["Timepoint"].idxmax()
    last_timepoint_df = last_timepoint_df.append(study_metadata.iloc[(volume_max)])

#Dropping Data and Presenting the DataFrame
last_timepoint_df = last_timepoint_df.drop(['Age_months','Metastatic Sites', 'Weight (g)'], 1)
last_timepoint_df
# Put treatments into a list for for loop (and later for plot labels)
# Create empty list to fill with tumor vol data (for plotting)
# Calculate the IQR and quantitatively determine if there are any potential outliers.     
    # Locate the rows which contain mice on each drug and get the tumor volumes
    # add subset 
    # Determine outliers using upper and lower bounds

#Creating Empty Lists and Empty DataFrame
tumor_vol_list = []
data_to_plot = []
outliers_df = pd.DataFrame(columns = ['Quartile 1', 'Quartile 3', 'IQR', 'Upper Outlier Bound',
                                      'Lower Outlier Bound', 'Upper Outliers','Lower Outliers', 'Drug Regimen'])

#Looping through each Specific Drug
for spdrug in specific_drug_list:
    
    #Creating DataFrame to get Tumor Volume, Getting List of Tumor Volume
    df = last_timepoint_df[last_timepoint_df['Drug Regimen'] == spdrug]
    tumor_vol_list = df['Tumor Volume (mm3)']
    
    #Get Summary Statistics, Creating Empty List for Potential Outliers
    q1 = np.percentile(tumor_vol_list, 25)
    q3 = np.percentile(tumor_vol_list, 75)
    iqr = q3-q1
    upper = q3 + (1.5* iqr)
    lower = q1 - (1.5 * iqr)
    u_out = []
    l_out = []
    
    #Looping through each Data per Specific Drug, finding Outliers and populating lists
    for i in tumor_vol_list:
        if i >= upper:
            u_out.append(i)
        elif i <= lower:
            l_out.append(i)
        
    # Populating the Empty Summary Data Frame
    outliers_df = outliers_df.append([{'Quartile 1' : q1, 'Quartile 3': q3, 'IQR':iqr, 'Upper Outlier Bound': upper,
                                      'Lower Outlier Bound': lower, 'Upper Outliers': f'{u_out}','Lower Outliers': f'{l_out}',
                                       'Drug Regimen': spdrug}])
    # Replacing Empty lists with 'None'
    outliers_df = outliers_df.replace(f'[]','None')
    data_to_plot.append(tumor_vol_list)

# Setting Drug Regimen as Index and Presenting Summary DF
outliers_df = outliers_df.set_index('Drug Regimen')
outliers_df
# Generate a box plot of the final tumor volume of each mouse across four regimens of interest
plt.boxplot(data_to_plot, labels = specific_drug_list)

# Setting Axes Labels and Titles
plt.xlabel('Drug Regimen')
plt.ylabel('Tumor Volume (mm3)')
plt.title('Final Tumor Volume Boxplot')
plt.tight_layout<jupyter_output><empty_output><jupyter_text>Line and Scatter Plots<jupyter_code># Generate a line plot of time point versus tumor volume for a mouse treated with Capomulin

#Filtering by Drug, Dropping Irrelevant Datam Grouping by Timepoint
capo_df = clean_df[clean_df['Drug Regimen'] == 'Capomulin']
capo_df = capo_df.drop(['Drug Regimen','Sex','Age_months','Metastatic Sites'], 1)
capo_line = capo_df.groupby(['Timepoint'])

#Plotting
plt.plot(capo_line.count().index, capo_line['Tumor Volume (mm3)'].mean())

#Setting Title, and Axes Labels
plt.title('Capomulin Timepoints vs. Tumor Size Line Plot')
plt.xlabel('Timepoints')
plt.ylabel("Tumor Volume (mm3)")

# Generate a scatter plot of mouse weight versus average tumor volume for the Capomulin regimen

#Grouping by Individual Mice
capo_scatter = capo_df.groupby(['Mouse ID'])

#Setting Axes
x_axis = capo_scatter['Weight (g)'].mean() 
y_axis = capo_scatter['Tumor Volume (mm3)'].mean()

#Plotting
plt.scatter(x_axis,y_axis)

#Setting Title and Axes Labels
plt.title('Capomulin Weight vs. Tumor Size Scatter Plot')
plt.xlabel('Weight of Mouse (g)')
plt.ylabel('Tumor Volume (mm3)')<jupyter_output><empty_output><jupyter_text>Correlation and Regression<jupyter_code># Calculate the correlation coefficient and linear regression model 
# for mouse weight and average tumor volume for the Capomulin regimen

#Finding Correlation Coefficients
(corr_coeff, _) = st.pearsonr(x_axis ,y_axis)

#Finding Line of Best Fit
(slope, intercept, _, _, _) = st.linregress(x_axis ,y_axis)
fit = slope * x_axis + intercept

#Plotting
fig, ax = plt.subplots()
ax.plot(x_axis, y_axis, linewidth=0, marker='o')
ax.plot(x_axis, fit, 'b--', linewidth = 1)

#Setting Title and Axes Labels
plt.title("Capomulin Weight vs. Tumor Size Linear Regression")
plt.xlabel('Weight of Mouse (g)')
plt.ylabel('Tumor Volume (mm3)')

plt.show()

print(f'The Correlation Coefficient is {corr_coeff.round(3)}')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09609494640122511, 'fraction_numerical': 0.004785604900459418, 'mean_word_length': 5.146470588235294, 'pattern_counts': {'":': 0, '<': 26, '<?xml version=': 0, '>': 26, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '238904', 'license': 'MIT', 'n_tokens_mistral': 3520, 'n_tokens_neox': 3242, 'n_words': 1120, 'path': 'pymaceuticals_starter.ipynb', 'repo_name': 'yohokr7/matplotlib-challenge', 'size': '10448'}