<jupyter_start><jupyter_text>Comparing Ground Truth with Gridded Products 1. The Weather Company<jupyter_code>%load_ext autoreload
%autoreload 2

import numpy as np
import pandas as pd
import geopandas as gpd
from datetime import date, datetime, timedelta
import requests
import os

%matplotlib inline
import matplotlib
import matplotlib.pyplot as plt
font = {'size'   : 12}
matplotlib.rc('font', **font)

pd.plotting.register_matplotlib_converters()
from lib.wx_utilities import *
# Some attributes of this site
lat, long, start, end = (41.17932065372352, -96.44152082846212, '2018-05-09', '2018-10-15')
# Read Device Data
df = pd.read_csv('dat/UNLCSP-3_daily.csv')
df['UTC'] = pd.to_datetime(df['UTC'], utc=True)
# Make a Backbone
df2 = pd.DataFrame()
df2['date'] = pd.date_range(start=start, end=end, freq="D", tz="UTC")
# Fetch TWC Gridded
wdf = pd.read_csv('dat/UNLCSP-3_twc.csv')
wdf = rename_wx_daily_for_merge(wdf)
wdf['date'] = pd.to_datetime(wdf['date'], utc=True)
# Merge everything
df2 = df2.merge(df, left_index=False, right_index=False, left_on='date',right_on='UTC', how='left')
df2 = df2.merge(wdf, left_on='date',right_on='date', how='left')
df2<jupyter_output><empty_output><jupyter_text>Compare Different Data Sources<jupyter_code># Temperature Compare
xlim = [pd.to_datetime(start), pd.to_datetime(end)]

fig, axes = plt.subplots(nrows=2, ncols=1, figsize=(12,8))

# Date versus Temp
line1 = axes[0].plot(df2.date, df2.Tmax, c='r', label = 'Mark')
line1 = axes[0].plot(df2.date, df2.Tmax_twc, c='r', ls='--', label = 'TWC')

line1 = axes[0].plot(df2.date, df2.Tmin, c='b', label = 'Mark')
line1 = axes[0].plot(df2.date, df2.Tmin_twc, c='b', ls='--', label = 'TWC')

ylim = axes[0].get_ylim()    

axes[0].set_xlim(xlim)    
axes[0].set_ylim(ylim)    
axes[0].legend()
axes[0].set_ylabel('Temperature (°C)')
axes[0].set_title('Weather Comparison')

# Date versus Temp Difference
line1 = axes[1].plot(df2.date, df2.Tmax_twc-df2.Tmax, c='r', label = 'Tmax')
line1 = axes[1].plot(df2.date, df2.Tmin_twc-df2.Tmin, c='b', label = 'Tmin')
line1, = axes[1].plot(xlim, [0, 0], c='b', lw=0.5, ls=':')

ylim = axes[1].get_ylim()    

axes[1].set_xlim(xlim)    
axes[1].set_ylim(ylim)    
axes[1].legend()
axes[1].set_ylabel('Temp Difference')
# Precip Compare
xlim = [pd.to_datetime(start), pd.to_datetime(end)]

fig, axes = plt.subplots(nrows=2, ncols=1, figsize=(12,8))

# Date versus Rain
line1 = axes[0].plot(df2.date, df2.Precip_1d, c='C0', label = 'Mark')
line1 = axes[0].plot(df2.date, df2.Precip_1d_twc*10, c='C1', ls='--', label = 'TWC')

ylim = axes[0].get_ylim()    

axes[0].set_xlim(xlim)    
axes[0].set_ylim(ylim)    
axes[0].legend()
axes[0].set_ylabel('Precip (mm)')
axes[0].set_title('Weather Comparison')

# Cumulative Rain
line1 = axes[1].plot(df2.date, np.cumsum(df2.Precip_1d), c='C0', label = 'Mark')
line1 = axes[1].plot(df2.date, np.cumsum(df2.Precip_1d_twc*10), c='C1', ls='--', label = 'TWC')

ylim = axes[1].get_ylim()    

axes[1].set_xlim(xlim)    
axes[1].set_ylim(ylim)    
axes[1].legend()
axes[1].set_ylabel('Precip (mm)')<jupyter_output><empty_output><jupyter_text>Exercises1. What is the mean difference and root mean square error of the daily temperature max and min?2. If harvest time was 1750 degree days, how many days different would that be between TWC and Mark?    This code may come in useful:    ```pythonfrom lib.biology import * The column names can be changed to use the TWC temperatures insteaddf2['GDD'] = GDD(df2, Tlo=10., Thi=44., method='II', Tmincol='Tmin', Tmaxcol='Tmax') df2['GDD'] = np.cumsum(df2['GDD'])    ```3. Open the file dat/CSP-3_2018.csv.  What are the GDD thresholds for the different growth stages?    This code may come in useful:    ```pythonobs = pd.read_csv('dat/CSP-3_2018.csv')obs['date'] = pd.to_datetime('2018-01-01') + obs['doy'].map(timedelta)    ```<jupyter_code>def error_metrics(vector1, vector2):
    mu = np.mean(vector1 - vector2)
    mae = np.mean(np.abs(vector1 - vector2))
    rmse = np.sqrt(((vector1 - vector2) ** 2).mean())
    return([mu, mae, rmse])

[error_metrics(df2.Tmax_twc, df2.Tmax), 
 error_metrics(df2.Tmin_twc, df2.Tmin)]<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.17847705981263512, 'fraction_numerical': 0.04780206581791977, 'mean_word_length': 5.347560975609756, 'pattern_counts': {'":': 0, '<': 13, '<?xml version=': 0, '>': 13, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '180057', 'license': 'Apache-2.0', 'n_tokens_mistral': 1834, 'n_tokens_neox': 1707, 'n_words': 353, 'path': 'materials/code/sensors/Notebook_1.ipynb', 'repo_name': 'agroimpacts/nmeo', 'size': '4163'}