<jupyter_start><jupyter_text>Concise Example Capacity Test using pvcaptestThis example performs the same test as the Complete Capacity Testing example, but uses the `run_test` function to apply the filters and then displays the filtering steps visually using the `scatter_filters` method. Imports<jupyter_code>import pandas as pd

# import captest as pvc
from captest import capdata as pvc
from bokeh.io import output_notebook, show

# uncomment below two lines to use cptest.scatter_hv in notebook
import holoviews as hv
hv.extension('bokeh')

#if working offline with the CapData.plot() method may fail
#run 'export BOKEH_RESOURCES=inline' at the command line before
#running the jupyter notebook

output_notebook()<jupyter_output><empty_output><jupyter_text>Load and Plot Measured DataWe begin by instantiating a `CapData` object, which we will use to load and store the measured data.  In this example we will calculate reporting conditions from the measured data, so we load and filter the measured data first.<jupyter_code>das = pvc.CapData('das')<jupyter_output><empty_output><jupyter_text>The `load_data` method by default will look for and attempt to load all files ending with '.csv' in a 'data' folder.  In this case we have a single file and provide the filename, so only the file specified is loaded.<jupyter_code>das.load_data(fname='example_meas_data.csv', source='AlsoEnergy')
das.set_regression_cols(power='-mtr-', poa='irr-poa-', t_amb='temp-amb-', w_vel='wind--')
das.agg_sensors(agg_map={'-inv-':'sum', 'irr-poa-':'mean', 'temp-amb-':'mean', 'wind--':'mean'},
                inv_sum_vs_power=False)
das.plot(marker='line', width=900, height=250, ncols=1)<jupyter_output><empty_output><jupyter_text>Filtering Measured DataThe `CapData` class provides a number of convience methods to apply filtering steps as defined in ASTM E2848.  The following section demonstrates the use of the more commonly used filtering steps to remove measured data points.<jupyter_code># Uncomment and run to copy over the filtered dataset with the unfiltered data.
das.reset_filter()
measured_filters = [
    (pvc.CapData.filter_sensors, (), {}),
    (pvc.CapData.filter_custom, (pd.DataFrame.dropna, ), {}),
    (pvc.CapData.filter_irr, (200, 2000), {}),
    (pvc.CapData.filter_outliers, (), {}),
    (pvc.CapData.fit_regression, (), {'filter':True, 'summary':False}),
    (pvc.CapData.rep_cond, (), {}),
    (pvc.CapData.filter_irr, (0.5, 1.5), {'ref_val':'self_val'}),
    (pvc.CapData.fit_regression, (), {}),
]
pvc.run_test(das, measured_filters)
das.get_summary()
das_scatter = das.scatter_filters()
das_scatter
das.timeseries_filters().opts(width=1200)<jupyter_output><empty_output><jupyter_text>Load and Filter PVsyst DataTo load and filter the modeled data, often from PVsyst, we simply create a new CapData object, load the PVsyst data, and apply the filtering methods as appropriate.<jupyter_code>sim = pvc.CapData('sim')<jupyter_output><empty_output><jupyter_text>To load pvsyst data we use the `load_data` method with the `load_pvsyst` option set to True.  By default the `load_data` method will search for a csv file that includes `pvsyst` in the filename in a `data` directory in the same directory as this file.  If you have saved the pvsyst file in a different location, you can use the `path` and `fname` arguments to load it.<jupyter_code>sim.load_data(load_pvsyst=True)
sim.column_groups
sim.set_regression_cols(power='real_pwr--', poa='irr-poa-', t_amb='temp-amb-', w_vel='wind--')
# Write over cptest.flt_sim dataframe with a copy of the original unfiltered dataframe
sim.reset_filter()
simulated_filters = [
    (pvc.CapData.filter_time, (), {'test_date':'10/11/1990', 'days':60}),
    (pvc.CapData.filter_irr, (200, 930), {}),
    (pvc.CapData.filter_pvsyst, (), {}),
    (pvc.CapData.filter_irr, (0.5, 1.5), {'ref_val':das.rc['poa'][0]}),
    (pvc.CapData.fit_regression, (), {}),
]
pvc.run_test(sim, simulated_filters)
sim.get_summary()
sim_scatter = sim.scatter_filters()
sim_scatter<jupyter_output><empty_output><jupyter_text>ResultsThe `get_summary` and `captest_results_check_pvalues` functions display the results of filtering on simulated and measured data and the final capacity test results comparing measured capacity to expected capacity, respectively.<jupyter_code>pvc.get_summary(das, sim)
das.rc
# the default hrs_req value is shown
das.print_points_summary(hrs_req=12.5)
pvc.captest_results_check_pvalues(sim, das, 6000, '+/- 7', print_res=True)<jupyter_output><empty_output><jupyter_text>The `overlay_scatters` function can be used to overlay the final scatter plots from scatter plots of all filtering steps produced above.<jupyter_code>pvc.overlay_scatters(das_scatter, sim_scatter)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11274716028607489, 'fraction_numerical': 0.011569204880100967, 'mean_word_length': 6.034023668639053, 'pattern_counts': {'":': 0, '<': 33, '<?xml version=': 0, '>': 33, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '535796', 'license': 'MIT', 'n_tokens_mistral': 1645, 'n_tokens_neox': 1551, 'n_words': 459, 'path': 'docs/examples/concise_capacity_test.ipynb', 'repo_name': 'matsuobasho/pvcaptest', 'size': '4754'}