<jupyter_start><jupyter_text>Figures appearing in the Algebraic Amplification paper at SIGMOD2020This notebook allows one to generate every synthetic data figure appearing in the Algebraic Amplification paper at SIGMOD 2020. Becacuse certain figures are time and compute intensive, each figure is generated from cached data. Each type of experiment has its own Python module (e.g. Fig_Timing, Fig_EndtoEnd_accuracy, etc.) imported at the top of this notebook. These modules define long lists of alternative parameters used for experiments. We choose sets of parameters with metaparameters CHOICE (which choice of parameters are chosen) and VARIANT (which variant for printing results is chosen). Other parameters define control variables like whether new data is created and whether figures are displayed or exported as PDF.<jupyter_code>import warnings
warnings.filterwarnings('ignore')
import sys
import time
sys.path.append("../")
import Fig_Timing, Fig_EndtoEnd_accuracy, Fig_Backtracking_Advantage, Fig_Time_VaryK, Fig_Homophily, Fig_Accuracy_Restarts, Fig_MHE_Robustness_d, Fig_MHE_Robustness_f, Fig_Scaling_Hrow, Fig_MHE_Variants_Scaling_Factor, Fig_MHE_Variants, Fig_MHE_Optimal_Lambda, Fig_Accuracy_Varyk, Fig_Timing_Accuracy
%matplotlib inline<jupyter_output><empty_output><jupyter_text>Figure 3  3a<jupyter_code>Fig_EndtoEnd_accuracy.run(108, 0, create_data=False, show_plot=True, create_pdf=True) ## Generates figure in the paper with over 100 repetition samples.

# Fig_EndtoEnd_accuracy.run(108, 4, create_data=True, show_plot=True, create_pdf=True)  # Smaller version that uses just 4 repetition samples.<jupyter_output><empty_output><jupyter_text>3b<jupyter_code>Fig_Timing.run(3, 6, show_plot=True, create_pdf=True)  # slide figure variant<jupyter_output><empty_output><jupyter_text>Figure 5  5a<jupyter_code>Fig_Backtracking_Advantage.run(31, 0, create_data=False, show_plot=True, create_pdf=True) ## Generates figure in the paper with over 1000 repetition samples.

# Fig_Backtracking_Advantage.run(31, 1, create_data=True, show_plot=True, create_pdf=True) # Smaller version which uses just 20 repetition rsamples.<jupyter_output><empty_output><jupyter_text>5b<jupyter_code>Fig_Scaling_Hrow.run(choice=12, variant=0, show_plot=False, create_pdf=True)  # slide figure variant (W^L, P_NB^{(L)})<jupyter_output><empty_output><jupyter_text>Figure 6  6a<jupyter_code>Fig_MHE_Variants.run(15, create_pdf=True)<jupyter_output><empty_output><jupyter_text>6b<jupyter_code>Fig_MHE_Variants_Scaling_Factor.run(17, create_pdf=True)   ## Generates figure in the paper with over 100 repetition samples.

# Fig_MHE_Variants_Scaling_Factor.run(18, create_data=True, create_pdf=True) # Smaller version that uses just 10 repetition rsamples.<jupyter_output><empty_output><jupyter_text>6cBelow figures appear more smooth (less variance) with additional experimental runs.<jupyter_code>Fig_MHE_Robustness_f.run(3, show_plot=True, create_pdf=True)    ## Generates figure in the paper with over 50 repetition samples.

# Fig_MHE_Robustness_f.run(9, create_data=True, show_plot=True, create_pdf=True) # Smaller version that uses just 5 repetition rsamples.<jupyter_output><empty_output><jupyter_text>6d<jupyter_code>Fig_MHE_Robustness_d.run(2, show_plot=True, create_pdf=True)   ## Generates figure in the paper with over 30 repetition samples.

# Fig_MHE_Robustness_d.run(8, create_data=True, show_plot=True, create_pdf=True)  # Smaller version that uses just 3 repetition rsamples.<jupyter_output><empty_output><jupyter_text>6e<jupyter_code>Fig_MHE_Optimal_Lambda.run(19, create_pdf=True)<jupyter_output><empty_output><jupyter_text>6f<jupyter_code>Fig_Timing_Accuracy.run(6, show_plot=True, create_pdf=True)<jupyter_output><empty_output><jupyter_text>6g<jupyter_code>Fig_Accuracy_Varyk.run(507, show_plot=True, create_pdf=True)<jupyter_output><empty_output><jupyter_text>6h<jupyter_code>Fig_Accuracy_Restarts.run(107, show_plot=True, create_pdf=True)   ## Generates figure in the paper with over 1000 repetition samples.

# Fig_Accuracy_Restarts.run(108, create_data=True, show_plot=True, create_pdf=True)  # Smaller version that uses just 10 repetition rsamples.<jupyter_output><empty_output><jupyter_text>6i<jupyter_code>Fig_Homophily.run(101, show_plot=True, create_pdf=True)

# Fig_Homophily.run(102, create_data=True, show_plot=True, create_pdf=True) ## Small Data<jupyter_output><empty_output><jupyter_text>6j<jupyter_code>Fig_EndtoEnd_accuracy.run(213, 0, show_plot=True, create_pdf=True)    ## Generates figure in the paper with over 100 repetition samples.

# Fig_EndtoEnd_accuracy.run(213, 1, create_data=True, show_plot=True, create_pdf=True) # Smaller test version which uses just 4 repetition rsamples.<jupyter_output><empty_output><jupyter_text>6k<jupyter_code>Fig_Timing.run(3,2, show_plot=True, create_pdf=True, show_pdf=False)<jupyter_output><empty_output><jupyter_text>6l<jupyter_code>Fig_Time_VaryK.run(607, show_plot=True, create_pdf=True) ## Generates figure in the paper with over 1000 repetition samples.

# Fig_Time_VaryK.run(608, create_data=True, show_plot=True, create_pdf=True) # Smaller test version that uses just 10 repetition rsamples.<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08681424446583254, 'fraction_numerical': 0.02444658325312801, 'mean_word_length': 9.37125748502994, 'pattern_counts': {'":': 0, '<': 69, '<?xml version=': 0, '>': 69, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '17', 'dir': 'jupyter-structured-clean-dedup', 'id': '562461', 'license': 'Apache-2.0', 'n_tokens_mistral': 1898, 'n_tokens_neox': 1773, 'n_words': 387, 'path': 'experiments_sigmod20/Figures_syntheticdata_sigmod20.ipynb', 'repo_name': 'northeastern-datalab/factorized-graphs', 'size': '5195'}