<jupyter_start><jupyter_text>CatEyes: Pandas Workflow Example_<NAME>_ In this notebook, I will give show you how you can easily combine your CatEyes workflow with [Pandas](https://pandas.pydata.org/) dataframes. This allows us to intuitively apply several CatEyes functions while keeping a nice overview over our data. It also allows us to seamlessly leverage all of Pandas' functionality designed to facilitate processing and plotting of time series data and tidy data in general. If you are looking for something simpler or don't want to use Pandas in your workflow, check out [minimal use example](https://github.com/DiGyt/cateyes/blob/main/example_minimal_use.ipynb).  Preprocessing Let's go straight ahead with installing the CatEyes library. If you have the library already installed in your environment, you can skip this cell.<jupyter_code># pip install the cateyes library from a bash command
!pip install git+https://github.com/DiGyt/cateyes.git -q<jupyter_output>Building wheel for cateyes (setup.py) ... [?25l[?25hdone
  Building wheel for nslr (setup.py) ... [?25l[?25hdone
  Building wheel for nslr-hmm (setup.py) ... [?25l[?25hdone<jupyter_text>Now, let's import all the libraries we need.<jupyter_code># import libraries
import warnings
warnings.filterwarnings('ignore')

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns

from cateyes import (plot_segmentation, plot_trajectory,
                    classify_nslr_hmm, classify_remodnav,
                    discrete_to_continuous, continuous_to_discrete)
from cateyes import sample_data_path<jupyter_output><empty_output><jupyter_text>Let's load our EyeTracking data. We load our example .csv straight into a pandas DataFrame.Another good idea is to reset the index to a pandas.Timedelta index right from the start. This will help us with time series operations like resampling. Afterwards lets check our data by calling the df.<jupyter_code># load the data and events
df = pd.read_csv(sample_data_path("example_data"))

# set the "Timestamp" field as index
df.index = pd.to_timedelta(df["Timestamp"], unit="S")
df = df.sort_index()

# show it
df<jupyter_output><empty_output><jupyter_text>Above, you can see that our DataFrame has three columns and one Timestamp index. The "Theta" column corresponds to the radian gaze rotation on the x axis, and the "Phi" column to the radian gaze rotation on the y axis. The "Timestamp" axis could theoretically be dropped as we have set it to our index already. Still, I often find it very helpful to also keep the time in pure seconds format. If you don't have anything corresponding to such a timestamp array, you can create one from the sampling frequency of your recording, using `cateyes.sfreq_to_times()`.Now, we can go on to preprocess that data. Preproccessing our time series data with Pandas is really easy. We can for example:- interpolate specific constant values- apply specific functions to our data (e.g. convert radian to degrees)- apply a boxcar median filter- crop our dataframe between two timepoints- resample our data to a constant sampling rateLet's just do this all right now, in only a few lines.<jupyter_code># interpolate all values that are either 0 or 1
for col in ["Theta", "Phi"]:
    mask = [i in (0, 1) for i in df[col]]
    df[col][mask] = np.nan
df = df.interpolate()

# convert radian data format to degree
df[["Theta", "Phi"]] = df[["Theta", "Phi"]].apply(np.rad2deg)

# Apply a median boxcar to our gaze data
boxcar_size = 3
df[["Theta", "Phi"]] = df[["Theta", "Phi"]].rolling(boxcar_size).median()
df[["Theta", "Phi"]] = df[["Theta", "Phi"]].fillna(method='backfill')

# crop the recording between 80 and 160 seconds
start = 80
stop = 160
df = df.loc[(df["Timestamp"] >= start) & (df["Timestamp"] <= stop)]

# resample to 200 Hz
sfreq = 200
df = df.resample(f"{1/sfreq}S").mean().interpolate()<jupyter_output><empty_output><jupyter_text>Now that we have our data in the format that we like to, we can think of adding events to our dataframe. Events might be any type of annotation that you've added to your eyetracking recording, for example external stimuli, triggers, trial indicators, and so on. In our example events, the participant followed a stimulus moving in different patterns. Let's load our events and print out the first few rows for demonstration.<jupyter_code># load and display the event data
event_df = pd.read_csv(sample_data_path("example_events"))
event_df.head()<jupyter_output><empty_output><jupyter_text>As you can see, our events have two columns, one for the Timestamp at which the event starts, and one for the corresponding Name of the event. Note that the timestamps in our events do not correspond to the sampling timestamps in our eyetracking data, neither before nor after preprocessing. Instead, they describe _discrete_ occurences during the course of our recording, which are not neccessarily aligned with the _continuous_ samples of our eyetracking data.This is a little unfortunate, as we usually want to have all our data in one place, in order to easily handle the data and prevent errors from possible disalignments. For this purpose, cateyes provides two functions to transalte between the _discrete_ and the _continuous_ format of our eyetracking data: `discrete_to_continuous` and `continuous_to_discrete`. The former function aligns discrete variables to a continuous time course, returning indices and values that match the continuous time course. The latter function reverts these indices and values back to a discrete time course, creating one timestamp and one value for each index. These functions are kind of "inverse" to each other, such that `discrete_to_continuous(times, continuous_to_discrete(times, indices, values))` will return `indices` and `values` again.Here we use `discrete_to_continuous` to map our events to our DataFrame, allowing us to append the events to our gaze series DataFrame.<jupyter_code># transform events and append them to the DataFrame
ev_idx, ev_names = discrete_to_continuous(df["Timestamp"], event_df["Timestamp"], event_df["Name"])
df["Event_Idx"] = ev_idx
df["Event_Name"] = ev_names
df<jupyter_output><empty_output><jupyter_text>As we can see, our events are now part of the DataFrame, which now contains all of our data.  Classification Now, we can go on and start classifying our eyetracking gaze series. The main purpose of CatEyes is to import different gaze classification algorithms and to put them into a uniform structure. Optimally, classifiers should work by calling `classify_xy(gaze_x, gaze_y, times)` and little else. By default, the function returns the segment indices and classes in _continuous_ format, but we could also return the discrete segments by passing `return_discrete=True`.Here, we will try out the NSLR-HMM and the REMoDNaV algorithms and append them to our DataFrame.<jupyter_code># add NSLR-HMM classification
segments, classes = classify_nslr_hmm(df["Theta"], df["Phi"], df["Timestamp"], optimize_noise=False)
df["NSLR_Segment"] = segments
df["NSLR_Class"] = classes<jupyter_output><empty_output><jupyter_text>Classifiers might require/allow additional arguments, which you can pass directly or through keyword arguments. For REMoDNaV, `px2deg` is a required argument, while `savgol_polyord` can be passed to REMoDNaV's internal preprocessing function.<jupyter_code># add remodnav class
segments, classes = classify_remodnav(df["Theta"], df["Phi"], df["Timestamp"], px2deg=1.,
                                      preproc_kwargs=dict(savgol_polyord=1), simple_output=True)
df["REMODNAV_Segment"] = segments
df["REMODNAV_Class"] = classes<jupyter_output><empty_output><jupyter_text>Now we've already got two distinct types of classifications. Let's check them out by plotting the first rows of our DataFrame.<jupyter_code># show dataframe
df.head()<jupyter_output><empty_output><jupyter_text>Of course, it looks nice, having everything in one place. But it's still hard to get some valuable information from simply looking at the table. For this purpose, cateyes introduces some simple but flexible plotting functions, allowing us to nicely visualize our gaze classifications.  Plotting The main function for visualization is `plot_segmentation`. This function plots the time course of our gaze data, while coloring each predicted gaze segment with the according class. It also allows us to plot any discrete event as a vertical line onto our gaze data, for example indicating trial starts in an eyetracking experiment.The ploting functions are intended to be as flexible as possible, with several optional parameters and a seamless integration into the matplotlib plotting framework. As seen below, the plot is treated as a standard matplotlib.pyplot element/axis, and can be changed accordingly.<jupyter_code># convert events and classifications to discrete format (for plotting)
dis_seg_class = continuous_to_discrete(df["Timestamp"],
                                       df["NSLR_Segment"],
                                       df["NSLR_Class"])
dis_events = continuous_to_discrete(df["Timestamp"], df["Event_Idx"],
                                    df["Event_Name"])

# create a matplotlib figure including plots for the x and y gaze array
fig, axes = plt.subplots(2, figsize=(15, 6), sharex=True)   
plot_segmentation(df["Theta"], df["Timestamp"], dis_seg_class, dis_events, ax=axes[0],
                  show_event_text=False, show_legend=False)
plot_segmentation(df["Phi"], df["Timestamp"], dis_seg_class, dis_events, ax=axes[1])

axes[0].set_ylabel("Theta (in degree)")
axes[0].set_xlim([80, 160])
axes[1].set_ylabel("Phi (in degree)")
axes[1].set_xlabel("Time in seconds");<jupyter_output><empty_output><jupyter_text>You could also use a custom discrete tuple of (times, values) to either color the segments or annotate the events in the plot. Below, we use events instead of the classification as color marker. For that purpose, we can pass a custom `color_dict`, mapping specific events to specific colors.<jupyter_code># create custom color dict
colors = ['y', 'g', 'b', 'r', 'c', 'm', 'k']
c_dict = {k:v for k, v in zip(set(df["Event_Name"]), colors)}

# plot it
plt.figure(figsize=(15, 3))
plot_segmentation(df["Theta"], df["Timestamp"], dis_events, color_dict=c_dict);<jupyter_output><empty_output><jupyter_text>The other main plot provided in cateyes visualizes the spatial instead of the temporal course of the gaze. Again there are many options to customize the plot and integrate it with matplotlib. Let's use this plot to show the classification results of REMoDNaV. In order to keep to plot visually appealing, we should probably not plot the entire recording, but only a short section. Fortunately, filtering is super easy with our combined pandas DataFrame. Let's just select all events during 3D hopping.<jupyter_code># create a filter mask for our pandas dataframe
mask = df["Event_Name"] == "hopping 3D"

# discretize the remodnav results
dis_seg_class = continuous_to_discrete(df["Timestamp"], df["REMODNAV_Segment"],
                                       df["REMODNAV_Class"])

# plot the spatial trajectory
plt.figure(figsize=(9, 6))
plot_trajectory(df[mask]["Theta"], df[mask]["Phi"], df[mask]["Timestamp"], dis_seg_class,
                show_legend=True, show_dots=False, show_clean=True, show_arrows=True,
                arrow_kwargs=dict(linestyle=(0, (5, 10))))
plt.grid(alpha=0.3)
plt.xlabel("X Rotation in °")
plt.ylabel("Y Rotation in °");<jupyter_output><empty_output><jupyter_text>Using pandas also allows us to use all of pandas internal plotting functions. We can e.g. compare Saccade classification between both our algorithms. Note that the main plot requires only three lines, everything else is embellishment.<jupyter_code># add a saccade mask and plot the data
df["NSLR_SP"] = (df["NSLR_Class"] == "Saccade") * 1
df["REMODNAV_SP"] = (df["REMODNAV_Class"] == "Saccade") * 1
axes = df[["Theta", "Phi", "NSLR_SP", "REMODNAV_SP"]].plot(subplots=True, legend=False)

# set labels, axes, etc...
axes[0].set_ylabel("Theta")
axes[1].set_ylabel("Phi")
axes[2].set_ylabel("NSLR")
axes[3].set_ylabel("REMODNAV")
axes[2].yaxis.tick_right()
axes[2].set_yticklabels(["", "No Saccade", "Saccade"])
axes[3].yaxis.tick_right()
axes[3].set_yticklabels(["", "No Saccade", "Saccade"])
plt.xlabel("Time");<jupyter_output><empty_output><jupyter_text>Going even further, we can use the [seaborn](https://seaborn.pydata.org/) plotting library, which is highly integrable with pandas. We can for example use it to compare the two classifiers in terms of their prediction frequencies.<jupyter_code>df_melt = df[["NSLR_Class", "REMODNAV_Class"]].melt(var_name="Classifier",
                                                    value_name="Prediction")
sns.countplot(x="Prediction", hue="Classifier", data=df_melt);<jupyter_output><empty_output><jupyter_text>Seaborn uses pandas dataframes to produce more complex plots, e.g. kernel density estimate (KDE) plots. Here we use them to investigate the spatial distribution of our classifiers when prediting post saccadic oscillations.<jupyter_code># kde plot for remodnav
mask = df["REMODNAV_Class"] == "PSO"
sns.kdeplot(df[mask]["Theta"], df[mask]["Phi"], color='b',
            shade=False, cmap="Blues", shade_lowest=False)

# kde plot for nslr
mask = df["NSLR_Class"] == "PSO"
sns.kdeplot(df[mask]["Theta"], df[mask]["Phi"], color='b',
            shade=False, cmap="Reds", shade_lowest=False)

# add legend/title
plt.title("Spatial Density of Gaze during PSO classification")
leg = plt.legend(["REMoDNaV", "NSLR"])
leg.legendHandles[0].set_color('blue')
leg.legendHandles[1].set_color('red')
plt.grid(alpha=0.3);<jupyter_output><empty_output><jupyter_text>Finally, we could use seaborn to visualize how our Theta and Phi gaze time courses relate to each other and to their rates of changes, separated by color according to NSLR-HMM's classification. This helps us get an intuition on how the algorithm works.<jupyter_code># calculate the rate of change for Theta and Phi
df["Theta_Change"] = df["Theta"] - df["Theta"].shift()
df["Phi_Change"] = df["Phi"] - df["Phi"].shift()

# create a seaborn pairplot
sns.pairplot(df[["Theta", "Phi", "Theta_Change", "Phi_Change", "NSLR_Class"]],
             hue="NSLR_Class");<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09530454196609224, 'fraction_numerical': 0.005651294216144562, 'mean_word_length': 5.210571923743501, 'pattern_counts': {'":': 0, '<': 66, '<?xml version=': 0, '>': 66, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 7, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '16', 'dir': 'jupyter-structured-clean-dedup', 'id': '585266', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 4314, 'n_tokens_neox': 4095, 'n_words': 1639, 'path': 'example_pandas_workflow.ipynb', 'repo_name': 'behinger/cateyes', 'size': '14338'}