<jupyter_start><jupyter_text>Download data from the USGS Water Services APIThe method `get_data` allows for programatically download of stream flow data from the USGS Water Services API. The input data for the method is the site identification and the range of dates. The inputs should be specified in a dictionary as it is shown below. Several options exist to pull data such as site id, HUC number (it returns all the stream gauges available in the HUC), and a coordinate bounding box. The example below focuses on pulling data for one site using the `site_id`. The user can opt for not saving the data on a file -- unlike the example where it is shown how to save the data in a csv. The `get_data` method always returns a list of pandas dataframes regardless of the number of sites requested. In the example below, that means that `flow_info` will be a list with one element, which is a dataframe.<jupyter_code>query = {}
query['id'] = '11370500'
query['start_date'] = '2010-01-01'
query['end_date'] = '2020-12-31'

# Call the get_data method. Note that the id_type has been set to 'sites'
flow_info = get_data(query, id_type='sites', save_data=True, path='/Users/galljf/Desktop/11370500_shiloh.csv')<jupyter_output>https://waterservices.usgs.gov/nwis/iv/?format=json&sites=11370500&startDT=2010-01-01&endDT=2020-12-31&parameterCd=00060&siteStatus=all
Success with data retrieval from API<jupyter_text>Inside `get_data`, other process take place in addition to just downloading the data. The data comes in `json` format, so the tool formats it and saves it as a dataframe. By default, the header of the stream flow data come as `Streamflow, ft&179;/s` as seen below. In addition, a timestamp is added along with coordinate information and qualifiers for the data. The latter are set by USGS and can be used for further data quality<jupyter_code>flow_info[0].head()
# Clean the data and make the date time column the index 
flow_info_temp = clean_data(flow_info[0], column_change={'Streamflow, ft&#179;/s':'discharge_cfs'}, keep_nan=True, timestamp_col='dateTime', no_data=-999999)
flow_info_temp.head()
flow_info_temp.drop(columns=['qualifiers','site_name','lat','long'], inplace=True)
flow_info_temp.head()
# Calculate the hydropower potential
turbine = calculate_potential(flow_info_temp, rated_flow=0, rated_power=0, head_input=500, system='pipe', flow_column='discharge_cfs')
turbine_power = flow_info_temp.copy()
turbine_power['power (MW)'] = turbine.power
turbine_power['efficiency (%)'] = turbine.effi_cal
turbine_power.head()
df = turbine_power.copy()
fig, ax1 = plt.subplots()

color = 'tab:red'
ax1.set_xlabel('Days')
ax1.set_ylabel('Flow rate (cu.ft/s)', color=color)
ax1.plot(df['discharge_cfs'], label="Flow rate", color=color)
ax1.tick_params(axis='y', labelcolor=color)

ax2 = ax1.twinx()  # instantiate a second axes that shares the same x-axis
color = 'tab:blue'
ax2.set_ylabel('Power (MW)', color=color)  # we already handled the x-label with ax1
ax2.plot(df['power (MW)'],'b-',label="Power", color=color)
ax2.tick_params(axis='y', labelcolor=color)
ax1.grid(True, axis='both', color='k',linestyle='--',alpha=0.4)
plt.title("Yearly flow data from USGS and potential power")
fig.tight_layout()  # otherwise the right y-label is slightly clipped
#plt.savefig(os.path.join('..','fig','usgs_twin_falls_flow_power.jpg'))
plt.show()
df = turbine_power.copy()
fig, ax1 = plt.subplots()

color = 'tab:red'
ax1.set_xlabel('Days')
ax1.set_ylabel('Flow rate (cu.ft/s)', color=color)
ax1.plot(df['discharge_cfs'], label="Flow rate", color=color)
ax1.tick_params(axis='y', labelcolor=color)

ax2 = ax1.twinx()  # instantiate a second axes that shares the same x-axis
color = 'tab:blue'
ax2.set_ylabel('Efficiency (%)', color=color)  # we already handled the x-label with ax1
ax2.plot(df['efficiency (%)'],'b-',label="Power", color=color)
ax2.tick_params(axis='y', labelcolor=color)
ax1.grid(True, axis='both', color='k',linestyle='--',alpha=0.4)
plt.title("Yearly flow data from USGS and potential power")
fig.tight_layout()  # otherwise the right y-label is slightly clipped
#plt.savefig(os.path.join('..','fig','usgs_twin_falls_flow_power.jpg'))
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1275023832221163, 'fraction_numerical': 0.025738798856053385, 'mean_word_length': 6.211340206185567, 'pattern_counts': {'":': 0, '<': 8, '<?xml version=': 0, '>': 8, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '39085', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 1461, 'n_tokens_neox': 1372, 'n_words': 433, 'path': 'examples/1.0-jugc-one_site_analysis.ipynb', 'repo_name': 'IdahoLabResearch/HydroGenerate', 'size': '4196'}