<jupyter_start><jupyter_text>Reproducing ResultsThis notebook contains the main experiments for the paper*Predicting Consumption Patterns with Repeated and Novel Events*It reproduces the main results from the paper for our method (tables 5 and 6).*PS: reddit_top is a very large dataset, so it might take a long time to load and compute.*  Downloading the data  You can find the data used [here](https://archive.ics.uci.edu/ml/datasets/Repeat+Consumption+Matrices)In order to use our data, Download and Unzip this file and place it in the project directory<jupyter_code>from models.mixture_model import train_mixture_model
from util.data_io import get_dataset

def mixture_model_results(dataset):
    """Method that trains and evalutes the Mixture Model for a particular dataset.
    Args:
        dataset: String that defines the subdirectry name where data should be. This subdirectory 
                 should be in a ./data/ directory and contain train.csv, validation.csv & test.csv
    """
    print dataset
    train, val, test = get_dataset(dataset)
    train_mixture_model(train, val, test, dataset_name=dataset)
    train_mixture_model(train, val, test, dataset_name=dataset, method='recall')
    print



# dataset can be: 'go_sf', 'go_ny', 'tw_oc', 'tw_ny', 'lastfm', 'reddit_sample', 'reddit_top'
mixture_model_results('go_sf')<jupyter_output>go_sf
2018-03-23 11:31:32,498 INFO --> Loading data for go_sf
2018-03-23 11:31:33,891 INFO --> Learning global mixing weights for all points
2018-03-23 11:31:34,045 INFO --> Finished EM on all data (global mix weights). Total time: 0 secs
2018-03-23 11:31:34,051 INFO --> Doing individual weights with 4 proc
2018-03-23 11:31:40,151 INFO --> User EM time: 6 secs -- 0.002 per user
2018-03-23 11:31:40,154 INFO --> Mean of mixing weights 	 {self:0.3409	 population:0.6591}
logP: -7.2562
recall: 0.4392
{'fraction_non_alphanumeric': 0.09213362068965517, 'fraction_numerical': 0.07058189655172414, 'mean_word_length': 4.398255813953488, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 10, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '105757', 'license': 'MIT', 'n_tokens_mistral': 657, 'n_tokens_neox': 568, 'n_words': 213, 'path': 'Experiments.ipynb', 'repo_name': 'UCIDataLab/repeat-consumption', 'size': '1856'}