<jupyter_start><jupyter_text>CoCoRaHS - Get station dataUse this notebook to get the data from all or a specific station between specified dates.<jupyter_code>%pylab inline
import requests
import io
import pandas as pd<jupyter_output><empty_output><jupyter_text>Select a county, or use all counties.In this example we want to get the rain gauges in Cook County, IL.<jupyter_code>counties = ["CK"] # Cook county
# counties = ["AD","AL","BD","BN","BR","BU","CH","CR","CS","CP","CN","CL","CY","CT","CE","CK","CW","CM","DK","DW","DG","DP","ED","EW","EF","FY","FD","FK","FL","GL","GR","GY","HM","HN","HR","HD","HY","IR","JK","JS","JF","JR","JD","JH","KN","KK","KD","KX","LK","LS","LW","LE","LV","LG","MCD","MCH","MCL","MA","MP","MD","MN","MH","MS","MC","ME","MR","MO","MY","MG","MU","OG","PR","PY","PT","PK","PP","PL","PM","RH","RC","RI","SC","SL","SG","SY","ST","SH","SK","SP","TZ","UN","VR","WB","WR","WS","WY","WH","WD","WL","WM","WN","WF"]<jupyter_output><empty_output><jupyter_text>Set the parameters. Here we want to get the data for the whole month of April 2013.<jupyter_code>params = {
   "ReportType": "Daily",
   "dtf": 1,
   "Format": "CSV",
   "State": "IL",
   "ReportDataType": "reportdate",
   "StartDate": "4/1/2013",
   "EndDate": "4/30/2013",
   "TimesInGMT": "False"
}
xml_responses = {}
for county in counties:
    params.update({
        "County": county
    })
    xml_responses[county] = requests.get("http://data.cocorahs.org/export/exportreports.aspx", params=params)<jupyter_output><empty_output><jupyter_text>Get the XML response text and convert it to string.Use `read_csv` from pandas to convert that to a dataframe, and concatenate all the county dataframes.<jupyter_code>df = pd.DataFrame()
for county in counties:
    df_county = pd.read_csv(io.StringIO(xml_responses[county].text), delimiter=',',header=0)
    df = pd.concat([df,df_county])<jupyter_output><empty_output><jupyter_text>Convert the `ObservationDate` column values to a pandas datetime objects, sort by date, and reset the index so that it follows the same order as the date column.<jupyter_code>df['ObservationDate'] = pd.to_datetime(df.ObservationDate)
df = df.sort_values(by='ObservationDate')
df = df.reset_index(drop=True)<jupyter_output><empty_output><jupyter_text>Convert the `TotalPrecipAmt` column from string to numeric values. In case of errors (when values are letters instead of numbers), force the converted value to be NaN by using `coerce`.<jupyter_code>df['TotalPrecipAmt'] = pd.to_numeric(df['TotalPrecipAmt'],errors='coerce')<jupyter_output><empty_output><jupyter_text>Check the output.<jupyter_code>df<jupyter_output><empty_output><jupyter_text>Get the list of all unique stations.<jupyter_code>stations = df['StationNumber'].unique()<jupyter_output><empty_output><jupyter_text>To check completeness of measurements of each station, print the station name and number of records.<jupyter_code>for station in stations:
    df_stn_1 = df.loc[(df['StationNumber']==station)]
    print(station+': '+str(len(df_stn_1)))<jupyter_output>IL-CK-64: 30
 IL-CK-87: 9
 IL-CK-175: 30
 IL-CK-168: 30
 IL-CK-208: 12
 IL-CK-163: 30
 IL-CK-197: 30
 IL-CK-196: 30
 IL-CK-148: 15
 IL-CK-193: 30
 IL-CK-82: 30
 IL-CK-180: 30
 IL-CK-192: 30
 IL-CK-153: 29
 IL-CK-152: 28
 IL-CK-154: 30
 IL-CK-211: 29
 IL-CK-143: 30
 IL-CK-145: 28
 IL-CK-173: 10
 IL-CK-137: 30
 IL-CK-149: 30
 IL-CK-95: 30
 IL-CK-69: 30
 IL-CK-75: 30
 IL-CK-61: 30
 IL-CK-42: 29
 IL-CK-49: 28
 IL-CK-36: 24
 IL-CK-14: 30
 IL-CK-12: 30
 IL-CK-53: 28
 IL-CK-51: 26
 IL-CK-74: 28
 IL-CK-50: 28
 IL-CK-88: 27
 IL-CK-121: 30
 IL-CK-131: 30
 IL-CK-107: 30
 IL-CK-139: 28
 IL-CK-81: 28
 IL-CK-98: 26
 IL-CK-94: 28
 IL-CK-106: 27
 IL-CK-129: 27
 IL-CK-100: 26
 IL-CK-204: 13
 IL-CK-63: 25
 IL-CK-177: 4
 IL-CK-202: 13
 IL-CK-170: 8
 IL-CK-182: 12
 IL-CK-32: 4
 IL-CK-91: 19
 IL-CK-187: 11
 IL-CK-179: 16
 IL-CK-132: 13
 IL-CK-66: 6
 IL-CK-214: 17
 IL-CK-199: 2
 IL-CK-178: 1
 IL-CK-206: 6
 IL-CK-83: 5
 IL-CK-218: 5<jupyter_text>If you want to select only a specific station, filter the dataframe to the station number.<jupyter_code>df_stn_1 = df.loc[(df['StationNumber']==' IL-CK-42')]
df_stn_1.head()<jupyter_output><empty_output><jupyter_text>Plot the station data.<jupyter_code>plt.figure(figsize=(10,3))
plt.plot(df_stn_1['ObservationDate'],df_stn_1['TotalPrecipAmt'])<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1948529411764706, 'fraction_numerical': 0.07238051470588236, 'mean_word_length': 5.2995658465991315, 'pattern_counts': {'":': 9, '<': 44, '<?xml version=': 0, '>': 44, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 3}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '249026', 'license': 'BSD-2-Clause', 'n_tokens_mistral': 1989, 'n_tokens_neox': 1702, 'n_words': 371, 'path': 'notebooks/raingauges/Cocorahs_get_station_data.ipynb', 'repo_name': 'amedendorp/precipitation-gradients', 'size': '4352'}