<jupyter_start><jupyter_text>Time domain averages (wide) Read and check the epochs<jupyter_code>import pandas as pd
from spudtr import epf
from spudtr import get_demo_df, DATA_DIR, P3_1500_FEATHER

epochs_df = get_demo_df(P3_1500_FEATHER)
eeg_channels = ['MiPf', 'MiCe', 'MiPa', 'MiOc']

epf.check_epochs(epochs_df, eeg_channels, epoch_id="epoch_id", time="time_ms")
epochs_df<jupyter_output>/home/turbach/miniconda3/envs/mckonda_spudtr_dev/lib/python3.6/site-packages/pyarrow/pandas_compat.py:752: FutureWarning: .labels was deprecated in version 0.24.0. Use .codes instead.
  labels, = index.labels<jupyter_text>Group by `time` to compute the time-domain average of all epochs and select columns of interest<jupyter_code>grand_wide = epochs_df.groupby(['time_ms']).mean()[eeg_channels]
grand_wide.columns.name = 'channel'
grand_wide<jupyter_output><empty_output><jupyter_text>Group by `time` and other columns to compute the average of subsets of epochs<jupyter_code>subsets_wide = epochs_df.groupby(["time_ms", "stim"]).mean()[eeg_channels]
subsets_wide.columns.name = "channel"
subsets_wide<jupyter_output><empty_output><jupyter_text>Time-domain averages (long)<jupyter_code>subsets_long = subsets_wide.stack()  # pivot the channel columns into one long column
subsets_long.name = "microvolts"
pd.DataFrame(subsets_long)<jupyter_output><empty_output><jupyter_text>Time interval measurmentsInterval measurments use the "slice-groupby-apply" pattern.  * slice the time interval rows* group by epoch_id and other tags* apply the measurment function to the data, e.g., pandas built-in or user-definedStart by doing the steps separately to verify.When the steps are right, chain them for compact expression.Example: single trial mean amplitude 1. Load the epochs<jupyter_code>eeg_channels = ["MiPf", "MiCe", "MiPa", "MiOc"]

epochs_df = get_demo_df(P3_1500_FEATHER).query('stim in ["target", "standard"]')
epf.check_epochs(epochs_df, eeg_channels, epoch_id="epoch_id", time="time_ms")
epochs_df<jupyter_output><empty_output><jupyter_text>2. (optional) select the data columns of interest or skip this and use them all.<jupyter_code>coi = ["epoch_id", "time_ms", "stim", "MiPf", "MiCe", "MiPa", "MiOc"]
mid = epochs_df[coi]  # select columns of interest

display(mid.head())
display(mid.tail())<jupyter_output><empty_output><jupyter_text>3. Slice the time interval data sample (rows) to measure and verify by inspection<jupyter_code>mid_300_500 = mid.query("time_ms >= 300 and time_ms <= 500")

display(mid_300_500.head())
display(mid_300_500.tail())
display(mid_300_500["time_ms"].min(), mid_300_500["time_ms"].max())<jupyter_output><empty_output><jupyter_text>4. **Group by** epoch_id, i.e., single trial, and other column labels to preserve them, and **apply** the built-in `mean()` function.**Note** the `time_ms` timestamps is just another column of data and also averaged in the interval.  **Note** `pandas.Dataframe` has dozens of [built-in stats functions](https://pandas.pydata.org/pandas-docs/stable/reference/frame.html?highlight=dataframe%20methodscomputations-descriptive-stats) besides mean: `max()`, `min()`, `std()`, `var()`, ...<jupyter_code>mid_300_500_mna = mid_300_500.groupby(["epoch_id", "stim"]).mean()

display(mid_300_500_mna.head(), mid_300_500_mna.tail())<jupyter_output><empty_output><jupyter_text>5. The epoch interval measurements are new data, re-label them appropriately.<jupyter_code># drop the no longer meaningful time_ms column
mid_300_500_mna = mid_300_500_mna.drop("time_ms", axis=1)

# describe the type of measurment and interval
mid_300_500_mna["measure"] = "mna"
mid_300_500_mna["interval"] = "300_500"

display(mid_300_500_mna.head(), mid_300_500_mna.tail())<jupyter_output><empty_output><jupyter_text>6. (optional) Export the measurements data<jupyter_code>mid_300_500_mna.reset_index().to_feather(DATA_DIR / "p3_mid_mna_300_500.feather")<jupyter_output><empty_output><jupyter_text>7. Chaining: All of the above, simplified by chaining. The results are verifiably identical.<jupyter_code>coi = ["epoch_id", "time_ms", "stim", "MiPf", "MiCe", "MiPa", "MiOc"]

# slice-groupby-apply
mid_300_500_mna_c  = (
    epochs_df[coi]
    .query("time_ms >= 300 and time_ms <= 500")
    .groupby(["epoch_id", "stim"])
    .mean()
    .drop("time_ms", axis=1)
)

# describe the type of measurment and interval
mid_300_500_mna_c["measure"] = "mna"
mid_300_500_mna_c["interval"] = "300_500"

# verify steps and chained agree
assert all(mid_300_500_mna_c == mid_300_500_mna)

# export
mid_300_500_mna_c.reset_index().to_feather(DATA_DIR / "p3_mid_mna_300_500.feather")<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12867647058823528, 'fraction_numerical': 0.044550173010380625, 'mean_word_length': 7.439781021897811, 'pattern_counts': {'":': 0, '<': 46, '<?xml version=': 0, '>': 46, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '453778', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 1869, 'n_tokens_neox': 1693, 'n_words': 363, 'path': 'docs/user_guide/gallery/gallery.ipynb', 'repo_name': 'turbach/spudtr', 'size': '4624'}