<jupyter_start><jupyter_text>01 - Exploring MoveDataFrameTo work with Pymove you need to import the data into our data structure: **MoveDataFrame**!**MoveDataFrame** is an abstraction that instantiates a new data structure that manipulates the structure the user wants. This is done using the Factory Method design pattern. This structure allows the interface to be implemented using different representations and libraries that manipulate the data.We have an interface that delimits the scope that new implementing classes should have. We currently have two concrete classes that implement this interface: **PandasMoveDataFrame** and **DaskMoveDataFrame** (under construction), which use Pandas and Dask respectively for data manipulation.It works like this: The user instantiating a MoveDataFrame provides a flag telling which library they want to use for manipulating this data.Now that we understand the concept and data structure of PyMove, **hands on!**--- MoveDataFrameA MoveDataFrame must contain the columns:- `lat`: represents the latitude of the point.- `lon`: represents the longitude of the point.- `datetime`: represents the date and time of the point.In addition, the user can enter several other columns as trajectory id. **If the id is not entered, the points are supposed to belong to the same path**.--- Creating a MoveDataFrameA MoveDataFrame can be created by passing a Pandas DataFrame, a list, dict or even reading a file. Look:<jupyter_code>import pymove as pm
from pymove import MoveDataFrame<jupyter_output><empty_output><jupyter_text>From a list<jupyter_code>list_data = [
    [39.984094, 116.319236, '2008-10-23 05:53:05', 1],
    [39.984198, 116.319322, '2008-10-23 05:53:06', 1],
    [39.984224, 116.319402, '2008-10-23 05:53:11', 1],
    [39.984224, 116.319402, '2008-10-23 05:53:11', 1],
    [39.984224, 116.319402, '2008-10-23 05:53:11', 1],
    [39.984224, 116.319402, '2008-10-23 05:53:11', 1]
]
move_df = MoveDataFrame(data=list_data, latitude="lat", longitude="lon", datetime="datetime", traj_id="id")
move_df.head()<jupyter_output><empty_output><jupyter_text>From a dict<jupyter_code>dict_data = {
    'lat': [39.984198, 39.984224, 39.984094],
    'lon': [116.319402, 116.319322, 116.319402],
    'datetime': ['2008-10-23 05:53:11', '2008-10-23 05:53:06', '2008-10-23 05:53:06']
}

move_df = MoveDataFrame(data=dict_data, latitude="lat", longitude="lon", datetime="datetime", traj_id="id")
move_df.head()<jupyter_output><empty_output><jupyter_text>From a DataFrame Pandas<jupyter_code>import pandas as pd

df = pd.read_csv('geolife_sample.csv', parse_dates=['datetime'])
move_df = MoveDataFrame(data=df, latitude="lat", longitude="lon", datetime="datetime")

move_df.head()<jupyter_output><empty_output><jupyter_text>From a file<jupyter_code>move_df = pm.read_csv('geolife_sample.csv')
move_df.head()<jupyter_output><empty_output><jupyter_text>Cool, huh? The default flag is Pandas. Look that:<jupyter_code>type(move_df)<jupyter_output><empty_output><jupyter_text>Let's try creating one with Dask!<jupyter_code>move_df = pm.read_csv('geolife_sample.csv', type_='dask')
move_df.head()
type(move_df)<jupyter_output><empty_output><jupyter_text>What's in MoveDataFrame?The MoveDataFrame stores the following information:<jupyter_code>orig_df = pm.read_csv('geolife_sample.csv')
move_df = orig_df.copy()<jupyter_output><empty_output><jupyter_text>1. The kind of data he was instantiated<jupyter_code>move_df.get_type()
move_df.columns
move_df.dtypes<jupyter_output><empty_output><jupyter_text>In addition to these attributes, we have some functions that allow us to: 1. View trajectory information<jupyter_code>move_df.show_trajectories_info()<jupyter_output>====================== INFORMATION ABOUT DATASET ======================

Number of Points: 217653

Number of IDs objects: 2

Start Date:2008-10-23 05:53:05     End Date:2009-03-19 05:46:37

Bounding Box:(22.147577, 113.548843, 41.132062, 121.156224)


=======================================================================<jupyter_text>2. View the number of users<jupyter_code>move_df.get_users_number()<jupyter_output><empty_output><jupyter_text>3. Transform our data to a. Numpy<jupyter_code>move_df.to_numpy()<jupyter_output><empty_output><jupyter_text>b. Dicts<jupyter_code>dict_data = move_df.to_dict()
dict_data.keys()<jupyter_output><empty_output><jupyter_text>c. DataFrames<jupyter_code>df = move_df.to_data_frame()
print(type(move_df))
print(type(df))
df<jupyter_output><class 'pymove.core.pandas.PandasMoveDataFrame'>
<class 'pandas.core.frame.DataFrame'><jupyter_text>4. And even switch from a Pandas to Dask and back again!<jupyter_code>new_move = move_df.convert_to('dask')
print(type(new_move))
move_df = new_move.convert_to('pandas')
print(type(move_df))<jupyter_output><class 'pymove.core.dask.DaskMoveDataFrame'>
<class 'pymove.core.pandas.PandasMoveDataFrame'><jupyter_text>5. You can also write files with<jupyter_code>move_df.write_file('move_df_write_file.txt')
move_df.to_csv('move_data.csv')<jupyter_output><empty_output><jupyter_text>6. Create a virtual grid<jupyter_code>move_df.to_grid(8)<jupyter_output><empty_output><jupyter_text>7. View the information of the last MoveDataFrame operation: operation name, operation time and memory use<jupyter_code>move_df.last_operation<jupyter_output><empty_output><jupyter_text>8. Get data bound box<jupyter_code>move_df.get_bbox()<jupyter_output><empty_output><jupyter_text>9. Create new columns: a. `tid`: trajectory id based on Id and datetime<jupyter_code>move_df.generate_tid_based_on_id_datetime()
move_df.head()<jupyter_output><empty_output><jupyter_text>b. `date`: extract date on datetime<jupyter_code>move_df.generate_date_features()
move_df.head()<jupyter_output><empty_output><jupyter_text>c. `hour`: extract hour on datetime<jupyter_code>move_df.generate_hour_features()
move_df.head()<jupyter_output><empty_output><jupyter_text>d. `day`: day of the week from datatime.<jupyter_code>move_df.generate_day_of_the_week_features()
move_df.head()<jupyter_output><empty_output><jupyter_text>e. `period`: time of day or period from datatime.<jupyter_code>move_df.generate_time_of_day_features()
move_df.head()<jupyter_output><empty_output><jupyter_text>f. `dist_to_prev`, `time_to_prev`, `speed_to_prev`:  create features of distance, time and speed to an GPS point P (lat, lon).<jupyter_code>move_df = orig_df.copy()
move_df.generate_dist_time_speed_features()
move_df.head()<jupyter_output><empty_output><jupyter_text>g. `dist_to_prev`, `dist_to_next`, `dist_prev_to_next` : three distance in meters to an GPS point P (lat, lon).<jupyter_code>move_df = orig_df.copy()
move_df.generate_dist_features()
move_df.head()<jupyter_output><empty_output><jupyter_text>h. `time_to_prev`, `time_to_next`, `time_prev_to_next` : three time in seconds to an GPS point P (lat, lon).<jupyter_code>move_df = orig_df.copy()
move_df.generate_time_features()
move_df.head()<jupyter_output><empty_output><jupyter_text>i. `speed_to_prev`, `speed_to_next`, `speed_prev_to_next` : three speed in meters by seconds to an GPS point P (lat, lon).<jupyter_code>move_df = orig_df.copy()
move_df.generate_speed_features()
move_df.head()<jupyter_output><empty_output><jupyter_text>j. `dist_to_prev`, `time_to_prev`, `speed_to_prev` : distance, time and speed from previous ponint<jupyter_code>move_df = orig_df.copy()
move_df.generate_dist_time_speed_features(inplace=False)<jupyter_output><empty_output><jupyter_text>K. `situation`:   column with move and stop points by radius.<jupyter_code>move_df = orig_df.copy()
move_df.generate_move_and_stop_by_radius()
move_df.head()<jupyter_output><empty_output><jupyter_text>9. Get time difference between max and min datetime in trajectory data.<jupyter_code>move_df.time_interval()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13088669318619273, 'fraction_numerical': 0.047221865776979344, 'mean_word_length': 8.201889020070839, 'pattern_counts': {'":': 0, '<': 126, '<?xml version=': 0, '>': 126, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '31', 'dir': 'jupyter-structured-clean-dedup', 'id': '577987', 'license': 'MIT', 'n_tokens_mistral': 3062, 'n_tokens_neox': 2771, 'n_words': 593, 'path': 'notebooks/01_Exploring_MoveDataFrame.ipynb', 'repo_name': 'JuniorNunes15/PyMove', 'size': '7793'}