<jupyter_start><jupyter_text>The Pst classThe `pst_handler` module contains the `Pst` class for dealing with pest control files.  It relies heavily on `pandas` to deal with tabular sections, such as parameters, observations, and prior information.<jupyter_code>from __future__ import print_function
import os
import numpy as np
import pyemu
from pyemu import Pst<jupyter_output><empty_output><jupyter_text>We need to pass the name of a pest control file to instantiate:<jupyter_code>pst_name = os.path.join("henry","pest.pst")
p = Pst(pst_name)<jupyter_output><empty_output><jupyter_text>All of the relevant parts of the pest control file are attributes of the `pst` class with the same name:<jupyter_code>p.parameter_data.head()
p.observation_data.head()
p.prior_information.head()<jupyter_output><empty_output><jupyter_text>A residual file (`.rei` or `res`) can also be passed to the `resfile` argument at instantiation to enable some simple residual analysis and weight adjustments.  If the residual file is in the same directory as the pest control file and has the same base name, it will be accessed automatically:<jupyter_code>p.res<jupyter_output><empty_output><jupyter_text>The `pst` class has some `@decorated` convience methods related to the residuals:<jupyter_code>print(p.phi,p.phi_components)<jupyter_output>1855.6874378297073 {'conc': 197.05822096106502, 'regul_m': 0.0, 'regul_p': 0.0, 'head': 1658.6292168686423}<jupyter_text>Some additional `@decorated` convience methods:<jupyter_code>print(p.npar,p.nobs,p.nprior)
print(p.par_groups,p.obs_groups)
print(type(p.par_names)) # all parameter names
print(type(p.adj_par_names)) # adjustable parameter names
print(type(p.obs_names)) # all observation names
print(type(p.nnz_obs_names)) # non-zero weight observations<jupyter_output><class 'list'>
<class 'list'>
<class 'list'>
<class 'list'><jupyter_text>The "control_data" section of the pest control file is accessible in the `Pst.control_data` attribute:<jupyter_code>print('jacupdate = {0}'.format(p.control_data.jacupdate))
print('numlam = {0}'.format(p.control_data.numlam))
p.control_data.numlam = 100
print('numlam has been changed to --> {0}'.format(p.control_data.numlam))<jupyter_output>jacupdate = 999
numlam = 10
numlam has been changed to --> 100<jupyter_text>The `Pst` class also exposes a method to get a new `Pst` instance with a subset of parameters and or obseravtions.  Note this method does not propogate prior information to the new instance:<jupyter_code>pnew = p.get(p.par_names[:10],p.obs_names[-10:])
print(pnew.prior_information)<jupyter_output>equation   obgnme    pilbl  weight      names
pilbl                                                                    
mult1    1.0 * log(mult1) = 0.000000  regul_m    mult1     1.0    [mult1]
kr01c01     1.0 * log(kr01c01) = 0.0  regul_p  kr01c01     1.0  [kr01c01]
kr01c02     1.0 * log(kr01c02) = 0.0  regul_p  kr01c02     1.0  [kr01c02]
kr01c03     1.0 * log(kr01c03) = 0.0  regul_p  kr01c03     1.0  [kr01c03]
kr01c04     1.0 * log(kr01c04) = 0.0  regul_p  kr01c04     1.0  [kr01c04]
kr01c05     1.0 * log(kr01c05) = 0.0  regul_p  kr01c05     1.0  [kr01c05]
kr01c06     1.0 * log(kr01c06) = 0.0  regul_p  kr01c06     1.0  [kr01c06]
kr01c07     1.0 * log(kr01c07) = 0.0  regul_p  kr01c07     1.0  [kr01c07]<jupyter_text>You can also write a pest control file with altered parameters, observations, and/or prior information:<jupyter_code>pnew.write("test.pst")<jupyter_output>/Users/jwhite/anaconda/lib/python3.5/site-packages/pandas/core/indexing.py:337: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
  self.obj[key] = _infer_fill_value(value)
/Users/jwhite/anaconda/lib/python3.5/site-packages/pandas/core/indexing.py:517: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
  self.obj[item] = s<jupyter_text>Some other methods in `Pst` include:<jupyter_code># add preferred value regularization with weights proportional to parameter bounds
pyemu.utils.helpers.zero_order_tikhonov(pnew)
pnew.prior_information
# add preferred value regularization with unity weights
pyemu.utils.helpers.zero_order_tikhonov(pnew,parbounds=False)
pnew.prior_information<jupyter_output><empty_output><jupyter_text>Some more `res` functionality<jupyter_code># adjust observation weights to account for residual phi components
#pnew = p.get()
print(p.phi, p.nnz_obs, p.phi_components)
p.adjust_weights_resfile()
print(p.phi, p.nnz_obs, p.phi_components)<jupyter_output>1855.6874378297073 36 {'conc': 197.05822096106502, 'regul_m': 0.0, 'regul_p': 0.0, 'head': 1658.6292168686423}
36.0 36 {'conc': 15.000000000000004, 'regul_m': 0.0, 'regul_p': 0.0, 'head': 21.0}<jupyter_text>adjust observation weights by an arbitrary amount by groups:<jupyter_code>print(p.phi, p.nnz_obs, p.phi_components)
grp_dict = {"head":100}
p.adjust_weights(obsgrp_dict=grp_dict)
print(p.phi, p.nnz_obs, p.phi_components)<jupyter_output>36.0 36 {'conc': 15.000000000000004, 'regul_m': 0.0, 'regul_p': 0.0, 'head': 21.0}
114.99999999999997 36 {'conc': 15.000000000000004, 'regul_m': 0.0, 'regul_p': 0.0, 'head': 99.99999999999997}<jupyter_text>adjust observation weights by an arbitrary amount by individual observations:<jupyter_code>print(p.phi, p.nnz_obs, p.phi_components)
obs_dict = {"h_obs01_1":25}
p.adjust_weights(obs_dict=obs_dict)
print(p.phi, p.nnz_obs, p.phi_components)<jupyter_output>114.99999999999997 36 {'conc': 15.000000000000004, 'regul_m': 0.0, 'regul_p': 0.0, 'head': 99.99999999999997}
138.82580701146588 36 {'conc': 15.000000000000004, 'regul_m': 0.0, 'regul_p': 0.0, 'head': 123.82580701146588}<jupyter_text>setup weights inversely proportional to the observation values<jupyter_code>p.adjust_weights_resfile()
print(p.phi, p.nnz_obs, p.phi_components)
p.proportional_weights(fraction_stdev=0.1,wmax=20.0)
print(p.phi, p.nnz_obs, p.phi_components)<jupyter_output>36.0 36 {'conc': 14.999999999999998, 'regul_m': 0.0, 'regul_p': 0.0, 'head': 21.0}
222.96996562151 36 {'conc': 194.30909581453392, 'regul_m': 0.0, 'regul_p': 0.0, 'head': 28.66086980697609}
{'fraction_non_alphanumeric': 0.11970306217135787, 'fraction_numerical': 0.09758738014228271, 'mean_word_length': 5.499497487437186, 'pattern_counts': {'":': 2, '<': 52, '<?xml version=': 0, '>': 54, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 10, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '14', 'dir': 'jupyter-structured-clean-dedup', 'id': '126664', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 2839, 'n_tokens_neox': 2429, 'n_words': 562, 'path': 'examples/pst_demo.ipynb', 'repo_name': 'jbellino-usgs/pyemu', 'size': '6466'}