<jupyter_start><jupyter_text>Observations and Insights Add your analysis here---The data from the research of drug affects on tumors for mice has only a few that has a stable effect in tumor size such as "Capomulin", "Ramicane", "Infubinol" and "Ceftamin". This can be seen by the change in low SEM for each drug. With equal population of genders we look at Capomulin as it shows a promising result in average tumor volume. The Capumlin seemed to have a regressive effect as well as stabilization around 35 days. It also seems to have a high correlation between weight and Average Tumor Vol. indicating it may have a lasting effect on the subject.<jupyter_code># Dependencies and Setup
import matplotlib.pyplot as plt
import pandas as pd
import scipy.stats as st
import numpy as np
from scipy.stats import linregress

# Study data files
mouse_metadata_path = "data/Mouse_metadata.csv"
study_results_path = "data/Study_results.csv"

# Read the mouse data and the study results
mouse_metadata = pd.read_csv(mouse_metadata_path)
study_results = pd.read_csv(study_results_path)
# len(study_results.dropna(how='any')['Mouse ID'])
# print(len(study_results['Mouse ID']))
# len(study_results['Mouse ID'].unique())
# Combine the data into a single dataset
mouseResults = pd.merge(mouse_metadata, study_results, on = 'Mouse ID')
# Display the data table for preview
mouseResults.head()
mouse_metadata.head()
study_results.head()
# Check the number of mice.
len(mouseResults['Mouse ID'].unique())
# Getting the duplicate mice by ID number that shows up for Mouse ID and Timepoint.
dupBool = mouseResults.duplicated(subset = ['Mouse ID', 'Timepoint'])
dup_mice = mouseResults.loc[dupBool]['Mouse ID'].unique()
dup_mice
# Optional: Get all the data for the duplicate mouse ID. 
duplicates = mouseResults.loc[mouseResults['Mouse ID']==dup_mice[0]]
duplicates
# Create a clean DataFrame by dropping the duplicate mouse by its ID.
# mouseResults.loc[mouseResults['Mouse ID']==dup_mice[0]]
# duplicates.index
noDup = mouseResults.drop(duplicates.index)
noDup
# Check the number of mice in the clean DataFrame.
len(noDup['Mouse ID'].unique())<jupyter_output><empty_output><jupyter_text>Summary Statistics<jupyter_code># Generate a summary statistics table of mean, median, variance, standard deviation, and SEM of the tumor volume for each regimen
df_frame= { 
    'Mean Tumor Volume' : noDup.groupby('Drug Regimen')['Tumor Volume (mm3)'].mean(),
    'Median Tumor Volume' : noDup.groupby('Drug Regimen')['Tumor Volume (mm3)'].median(),
    'Tumor Volume Variance' : noDup.groupby('Drug Regimen')['Tumor Volume (mm3)'].sum(),
    'Tumor Volume Std. Dev.' : noDup.groupby('Drug Regimen')['Tumor Volume (mm3)'].std(),
    'Tumor Volumne SEM' : noDup.groupby('Drug Regimen')['Tumor Volume (mm3)'].sem()}

DR_df = pd.DataFrame(df_frame)
# Use this straighforward method, create multiple series and put them all in a dataframe at the end.
DR_df
# Generate a summary statistics table of mean, median, variance, standard deviation, and SEM of the tumor volume for each regimen
drug_gp = noDup.groupby('Drug Regimen')
drug_df = drug_gp.agg({'Tumor Volume (mm3)':['mean','median','sum','std','sem']})
drug_df = drug_df.rename(columns = {'mean':'Mean Tumor Volume', 'median':'Median Tumor Volume',
                        'sum': 'Tumor Volume Variance', 'std': 'Tumor Volume Std. Dev.',
                        'sem': 'Tumor Volumne SEM'})
# Use method to produce everything with a single groupby function
drug_df<jupyter_output><empty_output><jupyter_text>Bar and Pie Charts<jupyter_code>totalTreatment = noDup.groupby('Drug Regimen')['Mouse ID'].count().sort_values(ascending = False).plot(kind = 'bar')
totalTreatment.set_ylabel("Number of Data Points")
plt.show()

# Generate a bar plot showing the total number of mice for each treatment throughout the course of the study using pandas. 
# Generate a bar plot showing the total number of mice for each treatment throughout the course of the study using pyplot.
tickLoc = range(len(DR_df['Mean Tumor Volume']))
plt.bar(tickLoc,noDup.groupby('Drug Regimen')['Mouse ID'].count())
plt.xticks(tickLoc, DR_df['Mean Tumor Volume'].index, rotation="vertical")
plt.xlabel('Drug Regimen')
plt.ylabel('Number of Data Points')
plt.show()
# Generate a pie plot showing the distribution of female versus male mice using pandas
# lbls = round(noDup['Sex'].value_counts()/len(noDup['Sex'])*100)
genderPlt = noDup['Sex'].value_counts().plot(kind='pie', autopct="%1.1f%%")
# Generate a pie plot showing the distribution of female versus male mice using pyplot
plt.pie(noDup['Sex'].value_counts(), labels = noDup['Sex'].unique(), autopct="%1.1f%%")
plt.ylabel('Sex')
plt.show()<jupyter_output><empty_output><jupyter_text>Quartiles, Outliers and Boxplots<jupyter_code># Calculate the final tumor volume of each mouse across each of the treatment regimens: 
# Start by getting the last (greatest) timepoint for each mouse
finalVol= noDup.groupby('Mouse ID')['Timepoint'].max()
# Merge this group df with the original dataframe to get the tumor volume at the last timepoint
finVol_df = pd.merge(finalVol, study_results, on = ['Mouse ID', 'Timepoint'], how = 'left')
finVol_df = pd.merge(finVol_df, mouse_metadata, on = ['Mouse ID'] )
finVol_df
# Put 4 treatment names into a list for use with a for loop (and later for plot labels)
treatment_list = ["Capomulin", "Ramicane", "Infubinol", "Ceftamin"]

# Create a empty list to fill with tumor vol data (for plotting) (hint: each element of the list will be series)
tumor_vol_list = []

# For each treatment in the list, calculate the IQR and quantitatively 
# determine if there are any potential outliers. 
for drug in treatment_list:
    tumor_vol_list.append(finVol_df.loc[finVol_df['Drug Regimen'] == drug]['Tumor Volume (mm3)'])
    # Locate the rows which contain mice on each drug and get the tumor volumes
    
    # add subset to tumor volume data list
    
    # Determine outliers using upper and lower bounds
    quartiles = tumor_vol_list[-1].quantile([.25,.5,.75])
    lowerq = quartiles[0.25]
    upperq = quartiles[0.75]
    iqr = upperq-lowerq
    lower_bound = lowerq - (1.5*iqr)
    upper_bound = upperq + (1.5*iqr)
    print(f"{drug}'s potential outliers: \
          {finVol_df.loc[(finVol_df['Tumor Volume (mm3)']<lower_bound) | (finVol_df['Tumor Volume (mm3)']>upper_bound) & (finVol_df['Drug Regimen']== drug)]['Tumor Volume (mm3)']}")
# Generate a box plot of the final tumor volume of each mouse across four regimens of interest
plt.boxplot(tumor_vol_list, labels = treatment_list, flierprops=({'markerfacecolor' : 'r', 'markersize' : '14'}))
plt.ylabel('Final Tumor Volume (mm3)')
plt.show()<jupyter_output><empty_output><jupyter_text>Line and Scatter Plots<jupyter_code>random_mouse = np.random.choice(noDup.loc[noDup['Drug Regimen'] == 'Capomulin']['Mouse ID'].unique())
rand_treat = noDup.loc[noDup['Mouse ID'] == random_mouse][['Tumor Volume (mm3)','Timepoint']]
plt.plot(rand_treat['Timepoint'], rand_treat['Tumor Volume (mm3)'])
plt.xlabel('Timepoint (Days)')
plt.ylabel('Tumor Volume (mm3)')
plt.title(f'Capomulin for {random_mouse}')
plt.show()

# Generate a line plot of time point versus tumor volume for a mouse treated with Capomulin

# Generate a scatter plot of mouse weight versus average tumor volume for the Capomulin regimen
capum_df = noDup.loc[noDup['Drug Regimen'] == 'Capomulin'].groupby('Mouse ID').mean()
scaplt = capum_df.plot(kind = 'scatter',x = 'Weight (g)', y = 'Tumor Volume (mm3)',s = 40)
scaplt.set_ylabel( 'Average Tumor Volume (mm3)')
plt.show()<jupyter_output><empty_output><jupyter_text>Correlation and Regression<jupyter_code># Calculate the correlation coefficient and linear regression model 
# for mouse weight and average tumor volume for the Capomulin regimen
xval = capum_df['Weight (g)']
yval = capum_df['Tumor Volume (mm3)']
corr = st.pearsonr(xval, yval)
print(f'The correlation between mouse weight and the average tumor volume is {round(corr[0],2)}')
slope, intercept, rvalue, pvalue, stderr = linregress(xval,yval)
regress_values = xval * slope + intercept

line_eq = "y = " + str(round(slope,2)) + "x + " + str(round(intercept,2))
scaplt = capum_df.plot(kind = 'scatter',x = 'Weight (g)', y = 'Tumor Volume (mm3)',s = 40)
scaplt.plot(xval,regress_values,"r-")
# scaplt.annotate(line_eq,(6,10),fontsize=10,color="red")
plt.show()<jupyter_output>The correlation between mouse weight and the average tumor volume is 0.84
{'fraction_non_alphanumeric': 0.12261806130903065, 'fraction_numerical': 0.007574860930287608, 'mean_word_length': 5.596409055425449, 'pattern_counts': {'":': 0, '<': 25, '<?xml version=': 0, '>': 25, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '636968', 'license': 'ADSL', 'n_tokens_mistral': 2869, 'n_tokens_neox': 2672, 'n_words': 882, 'path': 'Pymaceuticals/Pymaceuticals_Starter.ipynb', 'repo_name': 'timegnome/MatPlotLib-Challenge', 'size': '8449'}