<jupyter_start><jupyter_text>Rat Behavior DataFrame Docs Author: <NAME> NSDS lab The NSDS lab's rat reaching project collects a large variety of data pre and post data cleaning. This document is intended to highlight necessary datatypes, usage, and tips. More official documentation can be found at https://reachmaster.readthedocs.io/en/latest/ along with docstrings. Please note: Analysis_Utils.py must be in the same folder to currently work Data is saved in a .h5 file format and imported using pandas. Data is imported using code found here https://reachmaster.readthedocs.io/en/latest/software/extraction/extraction.html<jupyter_code>import numpy as np
import seaborn as sns
import pandas as pd
import pdb
import matplotlib.pyplot as plt
#import seaborn as sns
from Analysis_Utils import preprocessing_df as preprocessing
from Analysis_Utils import query_df
import mpl_toolkits.mplot3d.axes3d as p3
import matplotlib.animation as animation
def import_robot_data(df_path):
    df = pd.read_pickle(df_path)
    df = preprocessing(df)
    return df
import ast
df = import_robot_data('default_save_new_rightsample.pickle')




df['Date'][0]<jupyter_output><empty_output><jupyter_text>This is an example of one of the experimental variables (Date).  There are currently 23 variables, each composes a column inside the array.<jupyter_code>df.columns<jupyter_output><empty_output><jupyter_text>The dataframe is indexed by experiment number and within each block, an exposure index, which is arbitrary. The kinematic and experimental data are a 1:1 index match. Trials can be cut by using the r_start and r_stop values.<jupyter_code>def get_single_trial(df,date,session,rat):
    rr = df.loc[df['Date'] == date]
    rr = rr.loc[rr['S'] == session]
    new_df = rr.loc[rr['rat'] == rat]
    return new_df
ddv1=get_single_trial(df,'20','S1','RM14')

ddv2=get_single_trial(df,'20','S2','RM14')

ddv3=get_single_trial(df,'20','S3','RM16')
ddv3['SF'].values
ddv3['r_start'].to_list()[0][46] # total # trials
arshad_df=get_single_trial(df,'17','S4','RM16')
arshad_df['r_start'].values
arshad_df['r_stop'].values
brett_emily_df = get_single_trial(df,'19','S3','RM16')
brett_emily_df['r_start'].values
brett_emily_df['r_stop'].values
print(unique_pos,unique_pos1,unique_pos2)
# plot experimental positions
from mpl_toolkits.mplot3d import Axes3D
sns.set_style("whitegrid", {'axes.grid' : False})

fig = plt.figure(figsize=(6,6))

ax = Axes3D(fig)
ax.scatter(unique_pos[:,0],unique_pos[:,1],unique_pos[:,2], c='r',
               marker='o')
ax.scatter(unique_pos1[:,0],unique_pos1[:,1],unique_pos1[:,2], c='g',
               marker='x')
ax.scatter(unique_pos2[:,0],unique_pos2[:,1],unique_pos2[:,2], c='b',marker='v')

ax.set_xlabel('X CM')
ax.set_ylabel('Y CM')
ax.set_zlabel('Z CM')
plt.title('Handle locations in potentiometer space, S1, 9/25, RM14')<jupyter_output><empty_output><jupyter_text>Above, we have the data from a single experimental session (the third session of 9/25 using Rat 16). This is the building block from where building our analysis should stem (single experimental blocks). Variables from within a column can be called throug the 0 element of the values function. This returns a matrix (usually) of some length T.<jupyter_code>ddv1['SF'].values[0]<jupyter_output><empty_output><jupyter_text>Each variable is unique to the experiment, but some variables are catagorical that should be used for indexing.  rat	S	Date	dim	time	m_start	m_stop	SF	t_m	lick	x_p	y_p	z_p	x0	y0	z0	moving	RW	r_start	r_stop rat - starts with 'RMXX' , should be able to index and sort by date Date - starts with '1X', '2X', should be able to index and sort by date dim - 0, 3, should be able to index, sort by dim SF - this is a vector of the indices of correct trials (super confusing I know) it looks like [2, 5, 12], where 2, 5 and 12 are (you guessed it) successful trials in the experiment.  lick - vector of times rat licks [23.24, 24.32..] from potentiometer data x_p, y_p, z_p - vectors of potentiometer readings r_start, r_stop - These are vectors containing the indices of Trial start/stop frames (these are the indices used to slice kinematic data)  We will use use r_start and r_start, in combination with the SF indices and lick data, to obtain kinematic data from all behavioral epochs in our experiment.   Kinematic Data: to 3-D and Beyond! Our reaching data is in 3-D!! To read more about how this data is collected, check here at https://reachmaster.readthedocs.io/en/latest/software/preprocessing/preprocessing.htmlmodule-video_data.DLC.AV Below is an example dataframe from RM11, S1, 9/18 (18 Date)<jupyter_code>kinematic_df=pd.read_hdf('RM16_kinematics.h5')
kinematic_df.head()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10137857900318134, 'fraction_numerical': 0.024178154825026513, 'mean_word_length': 5.58659217877095, 'pattern_counts': {'":': 0, '<': 22, '<?xml version=': 0, '>': 22, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '122090', 'license': 'BSD-3-Clause-LBNL', 'n_tokens_mistral': 1625, 'n_tokens_neox': 1539, 'n_words': 523, 'path': 'software/ReachSplitter/Rat Behavior DataFrame Tutorial.ipynb', 'repo_name': 'BouchardLab/ReachMaster', 'size': '4721'}