<jupyter_start><jupyter_text>Trends: 1. For Capomulin, with a correlation of 0.84, there is a positive correlation between the mouse’ average tumor volume and its weight. 2. Capomulin and Ramicane: they have the most data points out of all the drugs present in the study. Their median and median tumor volume are the lowest among drugs tested. So they are the most successful drugs in reducing the tumor volume within the mice subjects. Next step is to focus on comparing these two drugs with more comprehensive datasets.  3. For one mouse, its tumor volume is not necessarily declining with timepoint increases though its long term trend is decreasing. Testers should be more patient during testing and provide more timepoint data points.           Pymaceuticals Inc.--- Analysis* Overall, it is clear that Capomulin is a viable drug regimen to reduce tumor growth.* Capomulin had the most number of mice complete the study, with the exception of Remicane, all other regimens observed a number of mice deaths across the duration of the study. * There is a strong correlation between mouse weight and tumor volume, indicating that mouse weight may be contributing to the effectiveness of any drug regimen.* There was one potential outlier within the Infubinol regimen. While most mice showed tumor volume increase, there was one mouse that had a reduction in tumor growth in the study.<jupyter_code># Dependencies and Setup
import matplotlib.pyplot as plt
import pandas as pd
import scipy.stats as st
import numpy as np

# Study data files
mouse_metadata_path = "data/Mouse_metadata.csv"
study_results_path = "data/Study_results.csv"

# Read the mouse data and the study results
mouse_metadata = pd.read_csv(mouse_metadata_path)
study_results = pd.read_csv(study_results_path)

# Combine the data into a single dataset, inner join
# Display the data table for preview
mouse_study = pd.merge(mouse_metadata, study_results, on="Mouse ID")
mouse_study.head(20)

# Checking the number of mice.
mouse_study["Mouse ID"].nunique()
# Getting the duplicate mice by ID number that shows up for Mouse ID and Timepoint. 
mouse_study_groupby_mouseid_timepoint = mouse_study.groupby(["Mouse ID", "Timepoint"])
mouse_study_groupby_mouseid_timepoint.count().head()
# Optional: Get all the data for the duplicate mouse ID. 
pd.set_option('display.max_rows', None)
mouse_study_groupby_mouseid_timepoint["Drug Regimen"].value_counts() # It shows g989 has duplicate time points 
mouse_study.loc[mouse_study["Mouse ID"] == "g989", :]
# Create a clean DataFrame by dropping the duplicate mouse by its ID.
mouse_study_cleaned = mouse_study.loc[mouse_study["Mouse ID"] != "g989", :]
mouse_study_cleaned.head(10)
# Checking the number of mice in the clean DataFrame.
mouse_study_cleaned["Mouse ID"].nunique()<jupyter_output><empty_output><jupyter_text>Summary Statistics<jupyter_code># Generate a summary statistics table of mean, median, variance, standard deviation, and SEM of the tumor volume for each regimen
# groupby Drug Regimen
mouse_study_cleaned_groubydrugregimen = mouse_study_cleaned.groupby(["Drug Regimen"])
mouse_study_cleaned_groubydrugregimen.count()
# Calculate mean, median, variance, standard deviation, and SEM of the tumor volume. 
# Assemble the resulting series into a single summary dataframe.

mean = mouse_study_cleaned_groubydrugregimen["Tumor Volume (mm3)"].mean()
median = mouse_study_cleaned_groubydrugregimen["Tumor Volume (mm3)"].median()
variance = mouse_study_cleaned_groubydrugregimen["Tumor Volume (mm3)"].var()
stdv = mouse_study_cleaned_groubydrugregimen["Tumor Volume (mm3)"].std()
sem = mouse_study_cleaned_groubydrugregimen["Tumor Volume (mm3)"].sem()

drugregimen_summary_df = pd.DataFrame({"Mean Tumor Volume": mean, "Median Tumor Volume": median, "Tumor Volume Variance": variance, "Tumor Volume Std. Dev.": stdv, "Tumor Volume Std. Err.": sem})
drugregimen_summary_df
# Generate a summary statistics table of mean, median, variance, standard deviation, and SEM of the tumor volume for each regimen
# Using the aggregation method, produce the same summary statistics in a single line

mouse_study_cleaned_groubydrugregimen.count()
drugregimen_summary_agg = mouse_study_cleaned_groubydrugregimen.agg(['mean','median','var','std','sem'])["Tumor Volume (mm3)"]
drugregimen_summary_agg<jupyter_output><empty_output><jupyter_text>Bar and Pie Charts<jupyter_code># Generate a bar plot showing the total number of measurements taken on each drug regimen using pandas.

mouse_study_cleaned_groubydrugregimen["Mouse ID"].count()
mouse_study_cleaned_groubydrugregimen["Mouse ID"].count().sort_values()
mouse_study_cleaned_groubydrugregimen["Mouse ID"].count().sort_values(ascending = False).plot(kind="bar", figsize=(5,4))
plt.ylabel("Total Number of Measurements")
plt.tight_layout()

# Generate a bar plot showing the total number of measurements taken on each drug regimen using using pyplot.

count = mouse_study_cleaned["Drug Regimen"].value_counts()
count.values

plt.bar(count.index.values, count.values)
plt.xticks(rotation=90)
plt.xlabel("Drug Regimen")
plt.ylabel("Total Number of Measurements")
# plt.tight_layout()
plt.show()
# Generate a pie plot showing the distribution of female versus male mice using pandas

mouse_study_cleaned["Sex"].value_counts()
mouse_study_cleaned["Sex"].value_counts().plot(kind="pie", autopct="%.1f%%", figsize=(5,4))
plt.show()
# Generate a pie plot showing the distribution of female versus male mice using pyplot

mouse_study_cleaned_sex_df = pd.DataFrame(mouse_study_cleaned["Sex"].value_counts())
mouse_study_cleaned_sex_df = mouse_study_cleaned_sex_df.rename(columns={"Sex": "Total Measurements per Sex"})
mouse_study_cleaned_sex_df = mouse_study_cleaned_sex_df.reset_index()
mouse_study_cleaned_sex_df = mouse_study_cleaned_sex_df.rename(columns={"index": "Sex"})
mouse_study_cleaned_sex_df

labels = mouse_study_cleaned_sex_df["Sex"]
sizes = mouse_study_cleaned_sex_df["Total Measurements per Sex"]
plt.pie(sizes, labels=labels, autopct="%.1f%%", startangle=0)
plt.ylabel("Sex")
plt.show()<jupyter_output><empty_output><jupyter_text>Quartiles, Outliers and Boxplots<jupyter_code># Calculate the final tumor volume of each mouse across four of the treatment regimens:  

# Capomulin, Ramicane, Infubinol, and Ceftamin
mouse_study_cleaned.head(10)
mouse_study_cleaned_4 = mouse_study_cleaned.loc[(mouse_study_cleaned["Drug Regimen"] == "Capomulin") | (mouse_study_cleaned["Drug Regimen"] == "Ramicane") | (mouse_study_cleaned["Drug Regimen"] == "Infubinol") | (mouse_study_cleaned["Drug Regimen"] == "Ceftamin"), :]
mouse_study_cleaned_4

# Start by getting the last (greatest) timepoint for each mouse
capturelasttimepoint = mouse_study_cleaned_4.groupby(["Mouse ID"]).max()['Timepoint']
capturelasttimepoint_df = pd.DataFrame(capturelasttimepoint)
capturelasttimepoint_df # Mouse ID is the index of the DataFrame

# Merge this group df with the original dataframe to get the tumor volume at the last timepoint
capturelasttimepoint_merge = pd.merge(capturelasttimepoint_df, mouse_study_cleaned, on=("Mouse ID","Timepoint"), how="left")
capturelasttimepoint_merge = capturelasttimepoint_merge.sort_values(by=['Drug Regimen'])
capturelasttimepoint_merge.head()
# Use function to calculate the IQR and quantitatively determine if there are any potential outliers. 

def chart(drug):
    tumorvolume = capturelasttimepoint_merge.loc[capturelasttimepoint_merge["Drug Regimen"] == drug, 'Tumor Volume (mm3)']
    quartiles = tumorvolume.quantile([.25,.5,.75])
    quartiles
    lowerq = quartiles[0.25]
    upperq = quartiles[0.75]
    iqr = upperq-lowerq   
    lower_bound = lowerq - (1.5*iqr)
    upper_bound = upperq + (1.5*iqr)
    print(f"We are checking on {drug}:")
    print(f"Values below {lower_bound} could be outliers.")
    print(f"Values above {upper_bound} could be outliers.")

# Put treatments into a list for for loop (and later for plot labels)

druglist = ["Capomulin", "Ramicane", "Infubinol", "Ceftamin"]
for drug in druglist:
    chart(drug)  
 
# # I am using Infubinol to test out my thought process, you can ignore this cell. But I am leaving it here in case I need to check it out years later. 

# tumorvolume_Infubinol = capturelasttimepoint_merge.loc[capturelasttimepoint_merge["Drug Regimen"] == "Infubinol", 'Tumor Volume (mm3)']
# tumorvolume_Infubinol
# quartiles_Infubinol = tumorvolume_Infubinol.quantile([.25,.5,.75])
# quartiles_Infubinol
# quartiles_Infubinol_median = quartiles_Infubinol.median()
# lowerq_Infubinol = quartiles_Infubinol[0.25]
# upperq_Infubinol = quartiles_Infubinol[0.75]
# iqr_Infubinol = upperq_Infubinol-lowerq_Infubinol  # about 11 
# lower_bound_Infubinol = lowerq_Infubinol - (1.5*iqr_Infubinol)
# upper_bound_Infubinol = upperq_Infubinol + (1.5*iqr_Infubinol)
# # print(f"we are checking on {drug}:")
# print(f"quartiles_Infubinol_median {quartiles_Infubinol_median}")
# print(f"lowerq_Infubinol {lowerq_Infubinol}")
# print(f"upperq_Infubinol {upperq_Infubinol}")


# print(f"Values below {lower_bound_Infubinol} could be outliers.")
# print(f"Values above {upper_bound_Infubinol} could be outliers.")
# plt.boxplot(tumorvolume_Infubinol)
# # I am using Infubinol to test out my thought process, you can ignore this cell. But I am leaving it here in case I need to check it out years later. 

# tumorvolume_Infubinol.sort_values()
# Generate a box plot of the final tumor volume of each mouse across four regimens of interest

cap_df = capturelasttimepoint_merge.loc[capturelasttimepoint_merge["Drug Regimen"] == "Capomulin", :]["Tumor Volume (mm3)"]
ram_df = capturelasttimepoint_merge.loc[capturelasttimepoint_merge["Drug Regimen"] == "Ramicane", :]["Tumor Volume (mm3)"]
inf_df = capturelasttimepoint_merge.loc[capturelasttimepoint_merge["Drug Regimen"] == "Infubinol", :]["Tumor Volume (mm3)"]
ceft_df = capturelasttimepoint_merge.loc[capturelasttimepoint_merge["Drug Regimen"] == "Ceftamin", :]["Tumor Volume (mm3)"]

data_to_plot = [cap_df, ram_df, inf_df, ceft_df]

fig1, ax1 = plt.subplots()
ax1.set_title('Tumors')
ax1.set_ylabel('Final Tumor Volume (mm3)')
ax1.set_xlabel('Drug Regimen')
outlier_style = dict(markerfacecolor='red', markersize=12)
ax1.boxplot(data_to_plot, labels=druglist, flierprops=outlier_style)

plt.savefig('boxplot')
plt.show()<jupyter_output><empty_output><jupyter_text>Line and Scatter Plots<jupyter_code># Generate a line plot of tumor volume vs. time point for a mouse treated with Capomulin
# Use loc condition to extract information of a specific mouseid

onemouseid = "l509"

onemouseid_df = mouse_study_cleaned.loc[mouse_study_cleaned["Mouse ID"] == onemouseid]
onemouseid_df = onemouseid_df.drop(columns=["Mouse ID", "Drug Regimen", "Sex", "Age_months", "Weight (g)", "Metastatic Sites"])
onemouseid_df = onemouseid_df.set_index("Timepoint")
onemouseid_df
onemouseid_df.plot(kind="line")
plt.title(f"Capomulin Treatment of mouse {onemouseid}")
plt.ylabel("Tumor Volume (mm3)")
plt.show()
plt.tight_layout()


# Generate a scatter plot of average tumor volume vs. mouse weight for the Capomulin regimen

mouse_study_cleaned_DrugRegimen_groupbyweight = mouse_study_cleaned.loc[mouse_study_cleaned["Drug Regimen"] == "Capomulin"].groupby(["Weight (g)", "Mouse ID"])
mouse_study_cleaned_DrugRegimen_groupbyweight.count()
mouse_study_cleaned_DrugRegimen_groupbyweight_tumorvolumemean = mouse_study_cleaned_DrugRegimen_groupbyweight["Tumor Volume (mm3)"].mean()
mouse_study_cleaned_DrugRegimen_groupbyweight_tumorvolumemean

mouse_study_cleaned_DrugRegimen_groupbyweight_tumorvolumemean_df = pd.DataFrame(mouse_study_cleaned_DrugRegimen_groupbyweight_tumorvolumemean)
mouse_study_cleaned_DrugRegimen_groupbyweight_tumorvolumemean_df = mouse_study_cleaned_DrugRegimen_groupbyweight_tumorvolumemean_df.reset_index()
plt.scatter(mouse_study_cleaned_DrugRegimen_groupbyweight_tumorvolumemean_df['Weight (g)'], mouse_study_cleaned_DrugRegimen_groupbyweight_tumorvolumemean_df['Tumor Volume (mm3)'])
plt.xlabel('Weight (g)')
plt.ylabel('Average Tumor Volume (mm3)')

plt.show()<jupyter_output><empty_output><jupyter_text>Correlation and Regression<jupyter_code># Calculate the correlation coefficient and linear regression model 
# for mouse weight and average tumor volume for the Capomulin regimen

corr=round(st.pearsonr(mouse_study_cleaned_DrugRegimen_groupbyweight_tumorvolumemean_df['Weight (g)'], mouse_study_cleaned_DrugRegimen_groupbyweight_tumorvolumemean_df['Tumor Volume (mm3)'])[0],2)
print(f"The correlation between mouse weight and average tumor volume is {corr}")
model=st.linregress(mouse_study_cleaned_DrugRegimen_groupbyweight_tumorvolumemean_df['Weight (g)'], mouse_study_cleaned_DrugRegimen_groupbyweight_tumorvolumemean_df['Tumor Volume (mm3)'])
model
mslope = 0.9544396890241044
bintercept = 21.55216053268502
y_values = mouse_study_cleaned_DrugRegimen_groupbyweight_tumorvolumemean_df['Weight (g)']*mslope+bintercept
plt.scatter(mouse_study_cleaned_DrugRegimen_groupbyweight_tumorvolumemean_df['Weight (g)'], mouse_study_cleaned_DrugRegimen_groupbyweight_tumorvolumemean_df['Tumor Volume (mm3)'])
plt.plot(mouse_study_cleaned_DrugRegimen_groupbyweight_tumorvolumemean_df['Weight (g)'], y_values,color="red")
plt.xlabel('Weight(g)')
plt.ylabel('Average Tumore Volume (mm3)')
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09609451425991422, 'fraction_numerical': 0.009933027315825118, 'mean_word_length': 7.481174218251436, 'pattern_counts': {'":': 7, '<': 25, '<?xml version=': 0, '>': 25, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '416143', 'license': 'CNRI-Python', 'n_tokens_mistral': 4651, 'n_tokens_neox': 4267, 'n_words': 1134, 'path': 'pymaceuticals_starter_with_plots.ipynb', 'repo_name': 'gongl1/matplotlib-challenge', 'size': '13289'}