<jupyter_start><jupyter_text>*grama* Analysis Demo---*grama* is a *grammar of model analysis*---a language for describing and analyzing mathematical models. Heavily inspired by [ggplot](https://ggplot2.tidyverse.org/index.html), `py_grama` is a Python package that implements *grama* by providing tools for defining and exploring models. This notebook illustrates how one can use *grama* to ***analyze a fully-defined model***.Note that you will need to install `py_grama`, a fork of `dfply`, and dependencies in order to run this notebook. See the [installation instructions](https://github.com/zdelrosario/py_grama) for details.<jupyter_code>### Setup
import grama as gr
import numpy as np
import pandas as pd
import seaborn as sns

X = gr.Intention()<jupyter_output><empty_output><jupyter_text>Quick Tour: Analyzing a model---*grama* separates the model *definition* from model *analysis*; once the model is fully defined, only minimal information is necessary for further analysis.As a quick demonstration, we import a fully-defined model provided with *grama*, and carry out a few analyses.<jupyter_code>from grama.models import make_cantilever_beam

md_beam = make_cantilever_beam()
md_beam.printpretty()<jupyter_output>model: Cantilever Beam

  inputs:
    var_det:
      t: [2, 4]
      w: [2, 4]
    var_rand:
      H: (+1) norm, {'loc': 500.0, 'scale': 100.0}
      V: (+1) norm, {'loc': 1000.0, 'scale': 100.0}
      E: (+0) norm, {'loc': 29000000.0, 'scale': 1450000.0}
      Y: (-1) norm, {'loc': 40000.0, 'scale': 2000.0}
    copula:
        Independence copula
  functions:
    cross-sectional area: ['w', 't'] -> ['c_area']
    limit state: stress: ['w', 't', 'H', 'V', 'E', 'Y'] -> ['g_stress']
    limit state: displacement: ['w', 't', 'H', 'V', 'E', 'Y'] -> ['g_disp']<jupyter_text>The method `printpretty()` gives us a quick summary of the model; we can see this model has two deterministic variables `w,t` and four random variables `H,V,E,Y`. All of the variables affect the outputs `g_stress, g_displacement`, while only `w,t` affect `c_area`. Since there are random variables, there is a source of *uncertainty* which we must consider when studying this model. Studying model behavior with uncertaintySince the model has sources of randomness (`var_rand`), we must account for this when studying its behavior. We can do so through a Monte Carlo analysis. We make decisions about the deterministic inputs by specifying `df_det`, and the `py_grama` function `gr.ev_monte_carlo` automatically handles the random inputs. Below we fix a nominal value `w = 0.5 * (2 + 4)`, sweep over values for `t`, and account for the randomness via Monte Carlo.<jupyter_code>## Carry out a Monte Carlo analysis of the random variables
df_beam_mc = \
    md_beam >> \
    gr.ev_monte_carlo(
        n=1e2, 
        df_det=gr.df_make( # Define deterministic levels
            w=0.5*(2 + 4), # Single value
            t=np.linspace(2.5, 3, num=10) # Sweep
        )
    )<jupyter_output>eval_monte_carlo() is rounding n...<jupyter_text>To help plot the data, we use `gr.tf_gather` to reshape the data, and `seaborn` to quickly visualize results.<jupyter_code>df_beam_wrangled = \
    df_beam_mc >> \
    gr.tf_gather("output", "y", ["c_area", "g_stress", "g_disp"])

g = sns.FacetGrid(df_beam_wrangled, col="output", sharey=False)
g.map(sns.lineplot, "t", "y")<jupyter_output><empty_output><jupyter_text>The mean behavior of the model is shown as a solid line, while the band visualizes the standard deviation of the model output. From this plot, we can see:- The random variables have no effect on `c_area` (there is no band)- Comparing `g_stress` and `g_displacement`, the former is more strongly affected by the random inputs, as illustrated by its wider uncertainty band.While this provides a visual description of how uncertainty affects our outputs, we might be interested in *how* the different random variables affect our outputs. Probing random variable effectsOne way to quantify the effects of random variables is through *Sobol' indices*, which quantify variable importance by the fraction of output variance "explained" by each random variable. Since distribution information is included in the model, we can carry out a *hybrid-point Monte Carlo* and analyze the results with two calls to `py_grama`.<jupyter_code>df_sobol = \
    md_beam >> \
    gr.ev_hybrid(n=1e3, df_det="nom", seed=101) >> \
    gr.tf_sobol()

df_sobol<jupyter_output>eval_hybrid() is rounding n...<jupyter_text>The indices should lie between `[0, 1]`, but estimation error can lead to violations. These results suggest that `g_stress` is largely insensitive to `E`, while `g_disp` is insensitive to `Y`. For `g_disp`, the input `V` contributes about twice the variance as variables `H,E`.To get a *qualitative* sense of how the random variables affect our model, we can perform a set of sweeps over random variable space with a *sinew* design. First, we visualize the design in the six-dimensional full input space.<jupyter_code>md_beam >> \
    gr.ev_sinews(n_density=50, n_sweeps=10, df_det="swp", skip=True) >> \
    gr.pt_auto()<jupyter_output>Estimated runtime for design with model (Cantilever Beam):
  0.0151 sec<jupyter_text>The `skip` keyword argument allows us to delay evaluating a model; this is useful for inspecting a design before running a potentially expensive calculation. The `pt_auto()` function automatically detects DataFrames generated by `py_grama` functions and constructs an appropriate visualization. This is provided for convenience; you are of course welcome (and encouraged!) to create your own visualizations of the data.Here we can see the sweeps cross the domain in straight lines at random starting locations. Each of these sweeps gives us a "straight shot" within a single variable. Visualizing the outputs for these sweeps will give us a sense of a single variable's influence, contextualized by the effects of the other variables.<jupyter_code>df_beam_sweeps = \
    md_beam >> \
    gr.ev_sinews(n_density=50, n_sweeps=10, df_det="swp")

df_beam_sweeps >> gr.pt_auto()<jupyter_output><empty_output><jupyter_text>Removing the keyword argument `skip` falls back on the default behavior; the model functions are evaluated at each sample, and `pt_auto()` adjusts to use this new data.Based on this plot, we can see:- The output `c_area` is insensitive to all the random variables; it changes only with `t, w`- As the Sobol' analyis above suggested `g_stress` is insensitive to `E`, and `g_displacement` is insensitive to `Y`- Visualizing the results shows that inputs `H,E` tend to 'saturate' in their effects on `g_displacement`, while `V` is linear over its domain. This may explain the difference in contributed variance- Furthermore both `t, w` seem to saturate in their effects on the two limit states---there are diminishing returns on making the beam taller or wider  Theory: The *grama* language---As a language, *grama* has both *objects* and *verbs*.   Objects---*grama* as a language considers two categories of objects:- **data** (`df`): observations on various quantities, implemented by the Python package `Pandas`- **models** (`md`): a function and complete description of its inputs, implemented by `py_grama`For readability, we suggest using prefixes `df_` and `md_` when naming DataFrames and models. Since data is already well-handled by Pandas, `py_grama` focuses on providing tools to handle models. A `py_grama` model has **functions** and **inputs**:  The method `printpretty()` gives a quick summary of the model's inputs and function outputs. Model inputs are organized into:|            | Deterministic                            | Random     || ---------- | ---------------------------------------- | ---------- || Variables  | `model.var_det`                          | `model.var_rand` || Parameters | `model.density.marginals[i].d_param`     | (Future*)  |- **Variables** are inputs to the model's functions  + **Deterministic** variables are chosen by the user; the model above has `w, t`  + **Random** variables are not controlled; the model above has `H, V, E, Y`- **Parameters** define random variables  + **Deterministic** parameters are currently implemented; these are listed under `var_rand` with their associated random variable  + **Random** parameters* are not yet implementedThe `outputs` section lists the various model outputs. The model above has `c_area, g_stress, g_displacement`.  Verbs---Verbs are used to take action on different *grama* objects. We use verbs to generate data from models, build new models from data, and ultimately make sense of the two.The following table summarizes the categories of `py_grama` verbs. Verbs take either data (`df`) or a model (`md`), and may return either object type. The prefix of a verb immediately tells one both the input and output types. The short prefix is used to denote the *pipe-enabled version* of a verb.| Verb Type | Prefix (Short)  | In   | Out   || --------- | --------------- | ---- | ----- || Evaluate  | `eval_` (`ev_`) | `md` | `df`  || Fit       | `fit_`  (`ft_`) | `df` | `md`  || Transform | `tran_` (`tf_`) | `df` | `df`  || Compose   | `comp_` (`cp_`) | `md` | `md`  |  Functional programming (Pipes)---`py_grama` provides tools to use functional programming patterns. Short-stem versions of `py_grama` functions are *pipe-enabled*, meaning they can be used in functional programming form with the pipe operator `>>`. These pipe-enabled functions are simply aliases for the base functions, as demonstrated below:<jupyter_code>df_base = gr.eval_nominal(md_beam, df_det="nom")
df_functional = md_beam >> gr.ev_nominal(df_det="nom")

df_base.equals(df_functional)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1080719794344473, 'fraction_numerical': 0.009151670951156812, 'mean_word_length': 4.294501905280349, 'pattern_counts': {'":': 0, '<': 29, '<?xml version=': 0, '>': 52, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '53030', 'license': 'MIT', 'n_tokens_mistral': 2916, 'n_tokens_neox': 2753, 'n_words': 1215, 'path': 'examples/demo/analysis_demo.ipynb', 'repo_name': 'QingmuDeng/py_grama', 'size': '9725'}