<jupyter_start><jupyter_text>Morning Creek Temp Study  Import Statements<jupyter_code>import datetime as dt
import time

from os import makedirs
import pandas as pd

from flowmeterAnalysis import readFiles
from flowmeterAnalysis import flowMonitor
from flowmeterAnalysis import plotting
from flowmeterAnalysis import basicMath<jupyter_output><empty_output><jupyter_text>File Locations<jupyter_code>homeDir = 'P:\\PW-WATER SERVICES\\TECHNICAL SERVICES\\Anna'
flowDir = homeDir + '\\2019\\Morning Creek Temp Study\\Data'
upstreamFile = homeDir + '\\MCTemptStudy_fmus.csv'
fmdataFile = homeDir + '\\FMdata.csv'
rainDir = homeDir + '\\2019\\Morning Creek Temp Study\\Rain'
hourlyFile = rainDir + '\\MCRG01_hourly_02012019-04302019.csv'
dailyFile = rainDir + '\\MCRG01_daily_02012019-04302019.csv'

saveDir = homeDir + '\\2019\\Morning Creek Temp Study\\'<jupyter_output><empty_output><jupyter_text>Settings and Variable Inputs<jupyter_code># EMPTY VARIABLES:
dfFlows = {}
basinDryWeather = {}
gageStorms = {}
stormsDict = {}
systemII = {}

# SETTINGS AND VARIABLES
analysisDates = (dt.datetime(2019,2,16),dt.datetime(2019,4,16))
rainThresh = 0.1<jupyter_output><empty_output><jupyter_text>CODE  read in rain files and flow monitor details as pandas dataframes* dfDaily is the daily rain data with the datetime as an index* dfHourly is the hourly rain data with the datetime as an index* dfDetails contains information about the rain gauge, diameter, basin footprint, basin area, etc. for each flow monitor* dfUpstream contains the immediate upstream flow monitors for each flow monitor<jupyter_code>dfDaily = readFiles.readRaincsv(
    filename = dailyFile)
dfHourly = readFiles.readRaincsv(
    filename = hourlyFile)
dfDetails = readFiles.readFMdetails(
    filename = fmdataFile)
dfUpstream = readFiles.readUpstreamFile(
    filename = upstreamFile)<jupyter_output><empty_output><jupyter_text>find all the flow csvs<jupyter_code>flow_folders,flow_txt,flow_csv = readFiles.findTextFiles(readDir=flowDir)<jupyter_output><empty_output><jupyter_text>construct basinDryWeatherDictThis is a python dictionary with the following keys and values:* Weekday: a dictionary containing the following keys:     * d/D: a numpy array containing the depth over diameter values    * Gross Q: a pandas series with datetimes as indices containing the gross flow rates    * DataFrame: a pandas dataframe with time as the index, dates for the columns, and values corresponding to measured flow rates* Weekend: a dictionary containing the following keys:     * d/D: a numpy array containing the depth over diameter values    * Gross Q: a pandas series with datetimes as indices containing the gross flow rates    * DataFrame: a pandas dataframe with time as the index, dates for the columns, and values corresponding to measured flow rates* Overall: a dictionary containing the following keys:     * BI: a float of the base infiltration, defined as the groundwater infiltration as a percentage of the overall mean flow rate    * d/D: a numpy array containing the depth over diameter values<jupyter_code>tic = time.process_time()
for fmData in flow_csv:
    if fmData.startswith('TFM'):
        fmname = fmData.split('_')[0] + '-' + fmData.split('_')[1]
    else:
        fmname = fmData.split('_')[0]
    flowFile = flowDir + "\\" + fmData
    dfFlow = readFiles.readFlowviewcsv(filename = flowFile)
    # if the data ends before the analysis date period, fill with nans
    if dfFlow.index[-1] < analysisDates[1]:
        dateList = list(basicMath.perdelta(
                start = dfFlow.index[-1] + dt.timedelta(minutes = 15), 
                end = analysisDates[1] + dt.timedelta(days = 1), 
                delta = dt.timedelta(minutes = 15)))
        dfNan = pd.DataFrame(
            data = {},
            index = dateList)
        for col in dfFlow.columns:
            dfNan[col] = float('NaN')
        dfFlow = pd.concat([dfFlow, dfNan])
    # ensure that only the analysis Dates are covered
    dfFlows[fmname] = dfFlow.loc[
         analysisDates[0]:analysisDates[1] + dt.timedelta(hours=23,minutes=45)
         , :]
    basinDryWeather[fmname] = flowMonitor.dryWeather(
        dfFlow = dfFlows[fmname], 
        fmname = fmname, 
        dfDailyRain = dfDaily,
        rainColName = 'Rain (in)',
        dfmDetails = dfDetails, 
        analysisDates = analysisDates,
        rainThresh = rainThresh)
toc = time.process_time()
print('Processing time = ' + str(toc - tic))<jupyter_output>Processing time = 5.959238200000016<jupyter_text>Create stacked area graphs of the upstream flows and downstream monitorThe downstream monitor should always be higher than the sum of the upstream flows<jupyter_code>tic = time.process_time()
folders, txt, csv = readFiles.findTextFiles(saveDir)
if 'Upstream Plots' not in folders:
    makedirs(saveDir + '\\Upstream Plots')
for fmname in dfFlows:
    usfmList = readFiles.findUpstreamFMs(
        df = dfUpstream, 
        fmname = fmname)
    if not usfmList:
        pass
    else:
        plotting.plotInstantUpstreamFlows(
            fmname = fmname, 
            dfFlows = dfFlows, 
            usfmList = usfmList,
            saveDir = saveDir + '\\Upstream Plots\\')
toc = time.process_time()
print('Processing time = ' + str(toc - tic))<jupyter_output><empty_output><jupyter_text>look into uptime during the study<jupyter_code>tic = time.process_time()
plotting.uptime(
    flowDict = dfFlows, 
    col = 'Q (MGD)', 
    saveDir = saveDir)
toc = time.process_time()
print('Processing time = ' + str(toc - tic))<jupyter_output>Processing time = 0.5928037999999844
{'fraction_non_alphanumeric': 0.09053571428571429, 'fraction_numerical': 0.020535714285714286, 'mean_word_length': 3.952254641909814, 'pattern_counts': {'":': 0, '<': 32, '<?xml version=': 0, '>': 31, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '222897', 'license': 'MIT', 'n_tokens_mistral': 1879, 'n_tokens_neox': 1692, 'n_words': 509, 'path': 'Morning Creek Temp Study.ipynb', 'repo_name': 'anskipper/jupyter-notebooks', 'size': '5600'}