<jupyter_start><jupyter_text>The following set of commands are those that are used for the results published in Kisley et al. 2021 (submitted).Note: These runs may not provide identical results due to the randomness of the K-fold cross validation data splitting (i.e. the data is split randomly each time into equal segments, and so slight variations in the KDE estimation may change the performance slightly). However, the purity and completeness per class will fall into the confidence intervals reported in Kisley et al. 2021 (submitted) 95% of the time. I.E. 95% of the time you run the following code, the performance (purity/completeness) will be within the ranges reported in the paper.The formatting and details of plots may appear different. All custom plotting is in the CustomPlotting Notebook.Runtime notes:To run locally, ensure you have followed the set up instruction in the README and have the data saved in the right location (data paths are in thex_data/data_consts.py).The following code is time consuming and is better ran as a background script. You may go to File > Download as .py. Then run it in the background. You may also update the CPU_COUNT  in thex_data/data_consts.py to a larger number, to take advantage of more parallel processsing.   Figure 1Completeness across full dataset.To generate completeness plot across the full dataset we use the nb=True flag, which does not drop rows even if they have NULL values for some of the columns considered. Each row need at least 1 valid value across the features considered. In this way, we may view the completeness across this set of features.<jupyter_code>import warnings
warnings.filterwarnings('ignore')
with warnings.catch_warnings():
    warnings.filterwarnings("ignore",category=DeprecationWarning)
%matplotlib inline  
from models.binary_model.binary_model import BinaryModel
from models.ind_model.ind_model import OvAModel
from models.multi_model.multi_model import MultiModel
mags = ["g_mag",  "r_mag", "i_mag", "z_mag", "y_mag",
        "W1_mag", "W2_mag",
        "J_mag", "K_mag", "H_mag"]
model = MultiModel(folds=10,
                   min_class_size = 50,    
                   transform_features = True,
                   cols = mags,
                   nb=True)  

model.visualize_data()<jupyter_output><empty_output><jupyter_text>Figure 2Distribution of transient classes.<jupyter_code>model = MultiModel(folds=10,
                   min_class_size = 50,    
                   transform_features = True,
                   cols = mags )  

model.visualize_data()<jupyter_output><empty_output><jupyter_text>Figures 5, 7, 14, 15Figure 5: Purity and completeness for binary modelFigures 7,14,15: Contain plots for all models; but this code will create the corresponding plots for the binary classifier<jupyter_code># Binary
binary_model = BinaryModel(folds = 10, 
                     min_class_size = 50,  
                     transform_features = True, 
                     cols = mags) 
binary_model.run_model()<jupyter_output><empty_output><jupyter_text>Figures 6, 7, 14, 15Figure 5: Purity and completeness for multi/OVA modelFigures 7,14,15: Contain plots for all models; but this code will create the corresponding plots for the OVA and Multi<jupyter_code>ova_model = OvAModel(folds = 10, 
                     min_class_size = 50,  
                     transform_features = True, 
                     cols = mags) 
ova_model.run_model()

multi_model = MultiModel(folds = 10, 
                     min_class_size = 50,  
                     transform_features = True, 
                     cols = mags) 
multi_model.run_model()<jupyter_output><empty_output><jupyter_text>Figures 8, 9, 10Completed using another Github repo; will be made public soon.  Figure 11Purity/completeness for 5 classes with vs without priors.<jupyter_code>multi_model = MultiModel(folds = 10, 
                     min_class_size = 50,  
                     transform_features = True, 
                     cols = mags,
                     priors = True,
                     classes= ["Unspecified Ia", "Ia-91bg", "Unspecified II", "Ibc", "TDE"],
                     lsst_test= True) #LSST_test flag just groups Ib, Ic, and Ib/c all into Ibc class name
multi_model.run_model()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.06525285481239804, 'fraction_numerical': 0.017245397343276625, 'mean_word_length': 3.111111111111111, 'pattern_counts': {'":': 0, '<': 21, '<?xml version=': 0, '>': 22, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '129845', 'license': 'MIT', 'n_tokens_mistral': 1288, 'n_tokens_neox': 1150, 'n_words': 453, 'path': 'notebooks/Publication Runs.ipynb', 'repo_name': 'marinakiseleva/thex_model', 'size': '4291'}