<jupyter_start><jupyter_text>Copyright (c) 2014-2021 National Technology and Engineering Solutions of Sandia, LLC. Under the terms of Contract DE-NA0003525 with National Technology and Engineering Solutions of Sandia, LLC, the U.S. Government retains certain rights in this software.     Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.  **Tutorial 2:** How to Create Trajectories from a Deliminated File<jupyter_code>import tracktable.examples.tutorials.tutorial_helper as tutorial<jupyter_output><empty_output><jupyter_text>PurposeThis notebook demonstrates how to create Tracktable trajectorys from a deliminated (e.g. csv, tsv, etc.) data file.  A data file must contain the following columns in order to be compatible with Tracktable:* **an identifier** that is unique to each object* **a timestamp*** **longitude*** **latitude**Both ordering and headers for these columns can vary, but they must exist in the file.  Each row of the data file should represent the information for a single trajectory point.**IMPORTANT:** Deliminated files must be **sorted by timestamp** to be compatible with Tracktable.*Note:* If you want to create individual Trajectory Point objects, but not assemble them into Trajectory objects, please see [Tutorial 1](Tutorial_01.ipynb).  Step 1: Set up a TrajectoryPointReader object.We will use the provided example data $^1$ for this tutorial.  For the sake of brevity, the function below executes the steps from [Tutorial 1](Tutorial_01.ipynb) to create a TrajectoryPointReader object.<jupyter_code>reader = tutorial.create_point_reader()<jupyter_output><empty_output><jupyter_text>Step 2: Create an AssembleTrajectoryFromPoints object. This will build trajectories from the individual points.<jupyter_code>from tracktable.applications.assemble_trajectories import AssembleTrajectoryFromPoints

builder = AssembleTrajectoryFromPoints()<jupyter_output><empty_output><jupyter_text>We must tell the builder to get the trajectory points from our reader.<jupyter_code>builder.input = reader<jupyter_output><empty_output><jupyter_text>*Additional Settings* How far apart (in km) should sequential points (with the same object ID) have to be before we consider them separate trajectories?  This parameter is optional and defaults to `None`.<jupyter_code>builder.separation_distance = 10 # km<jupyter_output><empty_output><jupyter_text>How far apart (in time) should sequential points (with the same object ID) have to be before we consider them separate trajectories?  This parameter is optional and defaults to 30 minutes.<jupyter_code>from datetime import timedelta

builder.separation_time = timedelta(minutes = 20)<jupyter_output><empty_output><jupyter_text>What is the minimum number of points that a trajectory must have?  Any trajectories assembled with fewer than this number will be discarded.  This parameter is optional and defaults to 2 points.<jupyter_code>builder.minimum_length = 5 # points<jupyter_output><empty_output><jupyter_text>Step 3: Assemble Trajectories from Point Data<jupyter_code>trajectories = list(builder)<jupyter_output>INFO:tracktable.applications.assemble_trajectoriesAssembleTrajectoryFromPoints:New trajectories will be declared after a separation of 10 distance units between two points or a time lapse of at least 0:20:00 (hours, minutes, seconds).
INFO:tracktable.applications.assemble_trajectoriesAssembleTrajectoryFromPoints:Trajectories with fewer than 5 points will be discarded.
INFO:tracktable.applications.assemble_trajectoriesAssembleTrajectoryFromPoints:Done assembling trajectories. 279 trajectories produced and 22 discarded for having fewer than 5 points.<jupyter_text>How many trajectories do we have?<jupyter_code>len(trajectories)<jupyter_output><empty_output><jupyter_text>Step 4: Accessing Trajectory Information For each `trajectory`, we can access the following information:* `trajectory.object_id`: a string identifier that is unique to each object* `trajectory.trajectory_id`: a string identifier that is mostly-unique to each trajectory, created by concatenating the object ID, start timestamp and end timestamp togetherThis is demonstrated below for our first ten trajectories.<jupyter_code>for trajectory in trajectories[:10]:
    object_id      = trajectory.object_id
    trajectory_id  = trajectory.trajectory_id
    
    print(f'Object ID: {object_id}')
    print(f'Trajectory ID: {trajectory_id}\n')<jupyter_output>Object ID: 367109000
Trajectory ID: 367109000_20200630000104_20200630002505

Object ID: 367484710
Trajectory ID: 367484710_20200630000243_20200630002642

Object ID: 367000140
Trajectory ID: 367000140_20200630000000_20200630005959

Object ID: 366999618
Trajectory ID: 366999618_20200630000000_20200630005949

Object ID: 367776270
Trajectory ID: 367776270_20200630000000_20200630005952

Object ID: 367022550
Trajectory ID: 367022550_20200630000000_20200630005919

Object ID: 367515850
Trajectory ID: 367515850_20200630000000_20200630005941

Object ID: 367531640
Trajectory ID: 367531640_20200630000000_20200630005950

Object ID: 338531000
Trajectory ID: 338531000_20200630000000_20200630005955

Object ID: 366516370
Trajectory ID: 366516370_20200630000000_20200630005940<jupyter_text>Step 5: Accessing Trajectory Point Information Let's look at just the first trajectory in our list:<jupyter_code>trajectory = trajectories[0]<jupyter_output><empty_output><jupyter_text>Trajectory points can be accessed in a trajectory object using list indexing.  So, we can get the first point in our trajectory as follows:<jupyter_code>trajectory_point = trajectory[0]<jupyter_output><empty_output><jupyter_text>The information from the required columns of the csv can be accessed for a single `trajectory_point` object as* **unique object identifier:** `trajectory_point.object_id`* **timestamp:** `trajectory_point.timestamp`* **longitude:** `trajectory_point[0]`* **latitude:** `trajectory_point[1]`The optional column information is available through the member variable `properties` as follows: `trajectory_point.properties['what-you-named-it']`.Below, we access all of the information stored in our `trajectory_point` object.<jupyter_code>object_id    = trajectory_point.object_id
timestamp    = trajectory_point.timestamp
longitude    = trajectory_point[0]
latitude     = trajectory_point[1]
heading      = trajectory_point.properties["heading"]
vessel_name  = trajectory_point.properties["vessel-name"]
eta          = trajectory_point.properties["eta"]

print(f'Unique ID: {object_id}')
print(f'Timestamp: {timestamp}')
print(f'Longitude: {longitude}')
print(f'Latitude: {latitude}')
print(f'Heading: {heading}')
print(f'Vessel Name: {vessel_name}')
print(f'ETA: {eta}\n')<jupyter_output>Unique ID: 367109000
Timestamp: 2020-06-30 00:01:04+00:00
Longitude: -74.2053
Latitude: 40.60922
Heading: 189.0
Vessel Name: OVERSEAS HOUSTON
ETA: 2020-06-30 13:40:04+00:00
{'fraction_non_alphanumeric': 0.06578783733366497, 'fraction_numerical': 0.07237905733117772, 'mean_word_length': 6.923152709359606, 'pattern_counts': {'":': 0, '<': 50, '<?xml version=': 0, '>': 50, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 8, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '618014', 'license': 'Unlicense', 'n_tokens_mistral': 2745, 'n_tokens_neox': 2229, 'n_words': 829, 'path': 'tutorial_notebooks/Tutorial_02.ipynb', 'repo_name': 'sandialabs/tracktable-docs', 'size': '8041'}