<jupyter_start><jupyter_text>Code to create a visualization for a single gap.<jupyter_code>import pandas as pd
import datetime as dt
from pyseas.contrib import plot_gap

%matplotlib inline<jupyter_output><empty_output><jupyter_text>Read in the data for the gap for gap_id `d380708e4e604791596e44d64c564f76`<jupyter_code>df_gaps = pd.read_csv("data/gaps_data.csv.zip", parse_dates=[2, 3])<jupyter_output><empty_output><jupyter_text>Get key attributes from the gap<jupyter_code>assert(df_gaps.shape[0] == 1)
gap = df_gaps.iloc[0]
gap_id = gap.gap_id
start_date = gap.gap_start
end_date = gap.gap_end
gap_in_ee = gap.gap_in_ee<jupyter_output><empty_output><jupyter_text>Read in the GFW AIS positions for the gap<jupyter_code>df_gap_positions_gfw = pd.read_csv("data/gap_positions_gfw.csv.zip", parse_dates=[4])<jupyter_output><empty_output><jupyter_text>Read in the Exact timestampth positions for the gapThis could be any secondary source of positions as long as it is formatted in the same way.<jupyter_code>df_gap_positions_ee = pd.read_csv("data/gap_positions_ee.csv.zip", parse_dates=[2])<jupyter_output><empty_output><jupyter_text>Read in the number of hourly pings for the GFW data  Read in the AIS performance metrics for this vesselAKA How frequently a vessel's AIS broadcasts with respect to reception quality, averaged over 2019**Warning: these are very rough, untested performance metrics so use with caution**<jupyter_code>df_hourly_gfw = pd.read_csv("data/hourly_gfw.csv.zip", parse_dates=[0])<jupyter_output><empty_output><jupyter_text>Calculate hourly pings for EE (or secondary AIS source) and join to GFW hourly pings<jupyter_code>df_performance = pd.read_csv("data/vessel_performance.csv.zip")
if gap_in_ee:
    df_hourly_ee = df_gap_positions_ee.groupby([pd.Grouper(key='timestamp',freq='H'), 'ssvid', 'gap_id']).size().reset_index(name='ee_positions')
    df_hourly_ee['hour'] = df_hourly_ee.timestamp.dt.hour
    df_hourly_ee['date'] = pd.to_datetime(df_hourly_ee.timestamp.dt.date)
    df_hourly_all = df_hourly_gfw.merge(df_hourly_ee, how='outer', on=['date', 'hour', 'ssvid', 'gap_id']).fillna(0, downcast='infer')
    df_hourly_all.drop(columns=['timestamp'], inplace=True)
else:
    df_hourly_all = df_hourly_gfw<jupyter_output><empty_output><jupyter_text>Plotting with *pyseas.plot_gap*  Plot without attribute table (table_type = 'none')<jupyter_code>fig, axes = plot_gap.plot_gap(gap_id, gaps_data=df_gaps, hourly_data=df_hourly_all, performance_data=df_performance, \
                                     positions_gfw=df_gap_positions_gfw, positions_ee=df_gap_positions_ee, show_all_gaps=False, \
                                     table_type='none')<jupyter_output>416000301
d380708e4e604791596e44d64c564f76<jupyter_text>Plot with a simplified attribute table (table_type = 'basic')This was designed for tracks that do not have Exact Earth (or other secondary AIS) data or when you do not want additional information about the secondary positions.<jupyter_code>fig, axes = plot_gap.plot_gap(gap_id, gaps_data=df_gaps, hourly_data=df_hourly_all, performance_data=df_performance, \
                                     positions_gfw=df_gap_positions_gfw, positions_ee=df_gap_positions_ee, show_all_gaps=False, \
                                     table_type='basic')<jupyter_output>416000301
d380708e4e604791596e44d64c564f76<jupyter_text>Plot with a full Exact Earth attribute table (table_type = 'ee')This was designed for tracks that have Exact Earth data and the user is trying to investigate the gap model.<jupyter_code>fig, axes = plot_gap.plot_gap(gap_id, gaps_data=df_gaps, hourly_data=df_hourly_all, performance_data=df_performance, \
                                     positions_gfw=df_gap_positions_gfw, positions_ee=df_gap_positions_ee, show_all_gaps=False, \
                                     table_type='ee')<jupyter_output>416000301
d380708e4e604791596e44d64c564f76<jupyter_text>Remove the Exact Earth data by omitting positions_ee parameter (or explicitly setting to None)<jupyter_code>fig, axes = plot_gap.plot_gap(gap_id, gaps_data=df_gaps, hourly_data=df_hourly_all, performance_data=df_performance, \
                                     positions_gfw=df_gap_positions_gfw, show_all_gaps=False, \
                                     table_type='basic')<jupyter_output>416000301
d380708e4e604791596e44d64c564f76<jupyter_text>Remove performance metrics by omitting performance_data (or explicitly setting to None)<jupyter_code>fig, axes = plot_gap.plot_gap(gap_id, gaps_data=df_gaps, hourly_data=df_hourly_all, \
                                     positions_gfw=df_gap_positions_gfw, positions_ee=df_gap_positions_ee, show_all_gaps=False, \
                                     table_type='ee')<jupyter_output>416000301
d380708e4e604791596e44d64c564f76<jupyter_text>Save to file using *out_filepath*Note: uncomment out the last portion of the funciton call below to save<jupyter_code>fig, axes = plot_gap.plot_gap(gap_id, gaps_data=df_gaps, hourly_data=df_hourly_all, performance_data=df_performance, \
                                     positions_gfw=df_gap_positions_gfw, positions_ee=df_gap_positions_ee, show_all_gaps=False, \
                                     table_type='basic')#, out_filepath="test.png")<jupyter_output>416000301
d380708e4e604791596e44d64c564f76
{'fraction_non_alphanumeric': 0.08516225848808853, 'fraction_numerical': 0.04539486025135997, 'mean_word_length': 4.8464912280701755, 'pattern_counts': {'":': 0, '<': 47, '<?xml version=': 0, '>': 47, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 12, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '527985', 'license': 'Apache-2.0', 'n_tokens_mistral': 2075, 'n_tokens_neox': 1821, 'n_words': 339, 'path': 'pyseas/doc/contrib/PlotGap.ipynb', 'repo_name': 'GlobalFishingWatch/pyseas', 'size': '5331'}