<jupyter_start><jupyter_text>Here's the shell for a simple temporal cross validation loop. It's based on Rayid's [magicloops](https://github.com/rayidghani/magicloops/blob/master/temporal_validate.py).First, we must set some parameters:<jupyter_code># start time of our data
start_time = datetime.strptime('2018-01-01', '%Y-%m-%d')

# last date of data including labels
# (this past Sunday!)
end_time = datetime.strptime('2018-06-30', '%Y-%m-%d')

# how far ahead we're predicting, e.g. '1' means the label 
# takes a 1 if an event takes place in the next month
# and a 0 otherwise
# unit: months
prediction_windows = [1]

# how often should we score?
# e.g. police might predict a year ahead (prediction window) every day (update window)
# unit: months
update_window = 1
df = pd.read_csv('temporal_CV.csv')
df<jupyter_output><empty_output><jupyter_text>convert to date format<jupyter_code>df['jail_entry'] = pd.to_datetime(df['jail_entry'], format = '%m/%d/%y')<jupyter_output><empty_output><jupyter_text>Dataframe for storing results<jupyter_code>results = pd.DataFrame(columns=['train_features_start_time', 'train_features_end_time', 
                                'train_label_start_time', 'train_label_end_time', 
                                'test_features_start_time', 'test_features_end_time',
                                'test__label_start_time', 'test_label_end_time', 
                                'accuracy'])
# the last test labels extend to the end of our data
test_label_end_time = end_time

for prediction_window in prediction_windows:
    
    # we'll start at the end to ensure the "freshest" possible data.
    # keep looping backward until there isn't enough time for two
    # prediction windows. (Remember, the train and test prediction
    # windows cannot overlap.)
    while (test_label_end_time >= start_time + 2 * relativedelta(months=+prediction_window)):
        
        # the prediction window equals the time between the start and end of the test label 
        test_label_start_time = test_label_end_time - relativedelta(months=+prediction_window)
        
        # the end of the train label window and test features window should precede the beginning of the test label window
        train_label_end_time = test_features_end_time = test_label_start_time - relativedelta(days=+1) 
        
        # the prediction window also equals the time between the start and end of the train label 
        train_label_start_time = train_label_end_time - relativedelta(months=+prediction_window)
        
        # the end of the train features should precede the beginning of the train labels
        train_features_end_time = train_label_start_time - relativedelta(days=+1)
        
        # for this example, we'll use all the data back to start_date
        train_features_start_time = test_features_start_time = start_time
        
        # only run if there's enough data for a full train label window
        while (train_label_start_time >= start_time):
            
            #train_label_start_time -= relativedelta(months=+prediction_window)
            
            # It's safer to split the data then generate features and labels
            raw_train_X = df[(df.jail_entry >= train_features_start_time) & (df.jail_entry <= train_features_end_time)]
            raw_train_y = df[(df.jail_entry >= train_label_start_time) & (df.jail_entry <= train_label_end_time)]
            raw_test_X = df[(df.jail_entry >= test_features_start_time) & (df.jail_entry <= test_features_end_time)]
            raw_test_y = df[(df.jail_entry >= test_label_start_time) & (df.jail_entry <= test_label_end_time)]
            
            # create the matrices we need
            
            # ensure that each entity is only represented at the appropriate times
            # e.g. B should not appear before 2/2/18
            
            # fit on train data
            # predict on test data
            # calculate accuracy
            # write results to the results dataframe
            
        test_label_end_time -= relativedelta(months=+update_window)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07826508359583233, 'fraction_numerical': 0.006784589290041192, 'mean_word_length': 3.0233918128654973, 'pattern_counts': {'":': 0, '<': 17, '<?xml version=': 0, '>': 19, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '331968', 'license': 'CC-BY-4.0', 'n_tokens_mistral': 1239, 'n_tokens_neox': 1188, 'n_words': 349, 'path': 'sources/curriculum/3_modeling_and_machine_learning/temporal-cross-validation/Temporal cross validation example.ipynb', 'repo_name': 'JackLaBarba/hitchhikers-guide', 'size': '4127'}