<jupyter_start><jupyter_text>Manual segmentation of gait dataHere we will demonstrate how to add manual segmentation labels from videos or any other source on top of pdkit's GaitTimeSeries.<jupyter_code># this is required for plotting in jupyter notebook
%matplotlib inline
from pdkit.gait_time_series import GaitTimeSeries
from pdkit.gait_processor import GaitProcessor
import numpy as np
import pandas as pd<jupyter_output><empty_output><jupyter_text>Let's load the data, from a standard cloudUPDRS format file<jupyter_code>ts = GaitTimeSeries().load_data('../tests/data/cloudupdrs_gait.csv')
ts.index
# instantiate the GaitProcessor
gp = GaitProcessor()<jupyter_output><empty_output><jupyter_text>Currently we add segmentation data using a python dictionary.- The keys represent the labels and the list of tuples represent the start and end of each section of that particular label.- The list of tuples can be of two types, one that uses integers for actual index locations on the timeseries, and another using timestamps.- If we use integers they will be automatically transformed into timestamps to match the time of the gait data.<jupyter_code>manual_segmentation_raw = {'walk': [(100, 450), (558, 962)],
                           'turn': [(68, 98), (455, 551)],
                           'stand': [(0, 65), (982, 1000)]}
# utility function to view the segmentation
gp.plot_segmentation_dictionary(ts, manual_segmentation_raw)
# if we are happy with the segmentation we can add it to the dataframe
segmented_df = gp.add_manual_segmentation_to_data_frame(ts, manual_segmentation_raw)
# as you can see we have a new column with segmentation data!
segmented_df.head()
# utility function to view the segmented dataframe
gp.plot_segmentation_data_frame(segmented_df)<jupyter_output><empty_output><jupyter_text>We can now do the exact same thing using datetime indices instead of numeric indexes.<jupyter_code>manual_segmentation_raw_timed = {'walk': [(np.datetime64('1970-01-01T00:00:02.124756000'), np.datetime64('1970-01-01T00:00:09.217316000')),
                                        (np.datetime64('1970-01-01T00:00:11.414276000'), np.datetime64('1970-01-01T00:00:19.439697000'))],
                               'turn': [(np.datetime64('1970-01-01T00:00:01.462250000'), np.datetime64('1970-01-01T00:00:02.082459000')),
                                        (np.datetime64('1970-01-01T00:00:09.320923000'), np.datetime64('1970-01-01T00:00:11.284546000'))],
                               'stand': [(np.datetime64('1970-01-01T00:00:00.000000000'), np.datetime64('1970-01-01T00:00:01.402985000')),
                                         (np.datetime64('1970-01-01T00:00:20.035370000'), np.datetime64('1970-01-01T00:00:20.409302000'))]}
gp.plot_segmentation_dictionary(ts, manual_segmentation_raw_timed)
segmented_df_timed = gp.add_manual_segmentation_to_data_frame(ts, manual_segmentation_raw_timed)
segmented_df_timed.head()
gp.plot_segmentation_data_frame(segmented_df_timed)<jupyter_output><empty_output><jupyter_text>Another interesting aspect of using numerical segmentation values instead of datetime is that they are not portable to resampled data.Let's resample the dataframe at 100hz and see how that works<jupyter_code>resampled = gp.resample_signal(ts)
filtered = gp.filter_data_frame(resampled)
segmented_resampled_df = gp.add_manual_segmentation_to_data_frame(resampled, manual_segmentation_raw)
gp.plot_segmentation_data_frame(segmented_resampled_df)<jupyter_output><empty_output><jupyter_text>Instead if we use the datetime segmentation we will get the real segmentation.<jupyter_code>segmented_timed_resampled_df = gp.add_manual_segmentation_to_data_frame(resampled, manual_segmentation_raw_timed)
gp.plot_segmentation_data_frame(segmented_timed_resampled_df)<jupyter_output><empty_output><jupyter_text>If you wanted to use numerical segmentation you have to provide the real segmentation values for the resampled data.<jupyter_code>manual_segmentation_filtered = {'walk': [(170, 997), (1155,1992)],
                               'turn': [(1004, 1152)],
                               'stand': [(0, 165), (2002, 2320)]}
gp.plot_segmentation_dictionary(filtered, manual_segmentation_filtered)
segmented_filtered = gp.add_manual_segmentation_to_data_frame(filtered, manual_segmentation_filtered)
gp.plot_segmentation_data_frame(segmented_filtered)<jupyter_output><empty_output><jupyter_text>Lastly let's save the segmented dataFor this we will use the builtin pandas saving functionality for DataFrames<jupyter_code>segmented_timed_resampled_df.to_csv('/tmp/filtered_segmented_data.csv')<jupyter_output><empty_output><jupyter_text>And now let's load it back<jupyter_code>seg_ts = GaitTimeSeries.load_data('/tmp/filtered_segmented_data.csv', format_file='segmented')
gp.plot_segmentation_data_frame(seg_ts)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09277926352602345, 'fraction_numerical': 0.07591030652129191, 'mean_word_length': 5.3889618922470435, 'pattern_counts': {'":': 0, '<': 37, '<?xml version=': 0, '>': 37, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 1, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '601429', 'license': 'MIT', 'n_tokens_mistral': 1814, 'n_tokens_neox': 1504, 'n_words': 348, 'path': 'notebooks/07 - Gait_manual_segmentation.ipynb', 'repo_name': 'gkroussos/pdkit', 'size': '4861'}