<jupyter_start><jupyter_text>First import soil moisture data from the master data sheet. We have 12 weeks of soil moisture data for 36 plots and 12 treatments.<jupyter_code>soil_moisture = pd.read_excel('2021 Data.xlsx', sheet_name = 'Soil Moisture' )
soil_moisture.head()<jupyter_output><empty_output><jupyter_text>Make the column names shorter<jupyter_code>soil_moisture = soil_moisture.rename(columns={'Plot Number': 'Plot'})
soil_moisture = soil_moisture.rename(columns={'Sample Date': 'Date'})
soil_moisture = soil_moisture.rename(columns={'Simp. Treatment': 'simp_trt'})
soil_moisture = soil_moisture.rename(columns={'Moisture Content': 'MC'})
soil_moisture.head()
ax = sns.boxplot(x = "Treatment", y = "MC", data = soil_moisture, hue = 'Short Trt Description', dodge = False)
plt.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.)
plt.savefig('trt_mc_box.jpg', bbox_inches='tight')<jupyter_output><empty_output><jupyter_text>The treatments don't look all that different. There is an outlier on the 30 in row treatment.<jupyter_code>ax = sns.boxplot(x = "Date", y = "MC", data = soil_moisture)
plt.savefig('date_bc_box.jpg', bbox_inches='tight')<jupyter_output><empty_output><jupyter_text>Function to filter the data by date<jupyter_code>def filterbydate (sampledate):
    sample_date_df = pd.DataFrame(soil_moisture[soil_moisture.Date == sampledate])
    return sample_date_df
filterbydate(2).head()<jupyter_output><empty_output><jupyter_text>Use that function to create box plots by date.<jupyter_code>sample_date = 1 
while sample_date <= max(soil_moisture.Date):
    date_df = filterbydate(sample_date)
    date_df.boxplot('MC', by = 'Treatment')
    plt.title('Moisture Content by Trt on Date {}'.format(sample_date))
    plt.suptitle("")
    plt.ylabel("Moisture Content, DB")
    sample_date = sample_date + 1
trt_means = pd.DataFrame(soil_moisture.groupby('Treatment')['MC'].describe()['mean'])
trt_means = trt_means.reset_index()
trt_means['mean'] = 100 * trt_means['mean']

ax = sns.barplot(x = 'Treatment', y = 'mean', data = trt_means)
ax.set_title('2021 Mean Soil Moisture Content')
ax.set(xlabel='Treatment', ylabel='Mean MC %')

#for annotating 
for p in ax.patches:
             ax.annotate("%.1f" % p.get_height(), (p.get_x() + p.get_width() / 2., p.get_height()),
                 ha='center', va='center', fontsize=10, color='black', xytext=(0, 5),
                 textcoords='offset points')
        
plt.savefig('MC_bar_trt.jpg', bbox_inches='tight')
block_means = pd.DataFrame(soil_moisture.groupby('Block')['MC'].describe()['mean'])
block_means = block_means.reset_index()
block_means['mean'] = 100 * block_means['mean']

ax = sns.barplot(x = 'Block', y = 'mean', data = block_means)
ax.set_title('2021 Mean Soil Moisture Content')
ax.set(xlabel='Block', ylabel='Mean MC %')

#for annotating 
for p in ax.patches:
             ax.annotate("%.1f" % p.get_height(), (p.get_x() + p.get_width() / 2., p.get_height()),
                 ha='center', va='center', fontsize=10, color='black', xytext=(0, 5),
                 textcoords='offset points')
        
plt.savefig('MC_bar_block.jpg', bbox_inches='tight')
day_means = pd.DataFrame(soil_moisture.groupby('Date')['MC'].describe()['mean'])
day_means = day_means.reset_index()
day_means['mean'] = 100 * day_means['mean']

ax = sns.barplot(x = 'Date', y = 'mean', data = day_means)
ax.set_title('2021 Mean Soil Moisture Content')
ax.set(xlabel='Sample Date', ylabel='Mean MC %')

#for annotating 
for p in ax.patches:
             ax.annotate("%.1f" % p.get_height(), (p.get_x() + p.get_width() / 2., p.get_height()),
                 ha='center', va='center', fontsize=10, color='black', xytext=(0, 5),
                 textcoords='offset points')
        
plt.savefig('MC_bar_date.jpg', bbox_inches='tight')<jupyter_output><empty_output><jupyter_text>Checking for Normality<jupyter_code>sns.kdeplot(data=soil_moisture, x="MC")
plt.title('Soil Moisture Kernel Density Plot')
plt.savefig('soil_moisture_kde.jpg', bbox_inches='tight')
plt.hist(x='MC', bins = 'auto', data = soil_moisture)
plt.title("Soil Moisture Content 2021")
plt.xlabel('Soil Moisture Content')
plt.ylabel('Frequency')
plt.savefig('Soil_MC_Histogram.jpg', bbox_inches='tight')
import scipy.stats as stats
stats.probplot(soil_moisture['MC'], dist="norm", plot=plt)
plt.show()
plt.savefig('soil_moisture_probplot', bbox_inches='tight')
pg.normality(soil_moisture, group='Treatment', dv='MC')
pg.normality(soil_moisture, group='Date', dv='MC')
pg.normality(soil_moisture, group='Block', dv='MC')
pg.normality(soil_moisture, group='Plot', dv='MC')<jupyter_output><empty_output><jupyter_text>Bartlett’s Test of Homogeneity of Variances<jupyter_code>soil_moisture['Treatment'].unique()
pip install pingouin
import pingouin as pg

# Bartlett's test in Python with pingouin:
pg.homoscedasticity(soil_moisture, dv='MC', 
                    group='Treatment',
                   method='bartlett')<jupyter_output><empty_output><jupyter_text>The treatments have equal variances.<jupyter_code>pg.homoscedasticity(soil_moisture, dv='MC', 
                    group='Date',
                   method='bartlett')<jupyter_output><empty_output><jupyter_text>The dates do not have equal variances.<jupyter_code>pg.homoscedasticity(soil_moisture, dv='MC', 
                    group='Block',
                   method='bartlett')<jupyter_output><empty_output><jupyter_text>Blocks have equal variances.  ANOVA<jupyter_code>from statsmodels.stats.anova import AnovaRM
aovrm = AnovaRM(soil_moisture, 'MC', subject = 'Plot', within = ['Date'])
res = aovrm.fit()
print(res)<jupyter_output>Anova
=====================================
     F Value   Num DF  Den DF  Pr > F
-------------------------------------
Date 182.8609 11.0000 385.0000 0.0000
=====================================
{'fraction_non_alphanumeric': 0.169103733012214, 'fraction_numerical': 0.015310510923791501, 'mean_word_length': 5.09433962264151, 'pattern_counts': {'":': 0, '<': 41, '<?xml version=': 0, '>': 41, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '311911', 'license': 'MIT', 'n_tokens_mistral': 2251, 'n_tokens_neox': 2080, 'n_words': 386, 'path': '_data/Soil Moisture Analysis.ipynb', 'repo_name': 'gabbymyers/516X-Project', 'size': '5813'}