<jupyter_start><jupyter_text>Feature: Meta-Features Using Pairwise Diffs/Ratios Generate additional meta-features derived from pairwise differences and ratios of most important features (helpful for tree-based models).  Imports This utility package imports `numpy`, `pandas`, `matplotlib` and a helper `kg` module into the root namespace.<jupyter_code>from pygoose import *
import itertools<jupyter_output><empty_output><jupyter_text>Config Automatically discover the paths to various data folders and compose the project structure.<jupyter_code>project = kg.Project.discover()<jupyter_output><empty_output><jupyter_text>Identifier for storing these features on disk and referring to them later.<jupyter_code>feature_list_id = 'meta_pairwise_interactions'<jupyter_output><empty_output><jupyter_text>Feature lists containing the most important features.<jupyter_code>feature_lists = [
    # Place feature lists here.
]<jupyter_output><empty_output><jupyter_text>Names of the features to use for pairwise interactions.<jupyter_code>most_important_features = [
    # Place features here.
]<jupyter_output><empty_output><jupyter_text>Read data<jupyter_code>df_train, df_test, _ = project.load_feature_lists(feature_lists)<jupyter_output><empty_output><jupyter_text>Build features<jupyter_code>feature_names = []
for a, b in itertools.combinations(most_important_features, 2):
    diff_feature_name = f'diff_{a}_{b}'
    df_train[diff_feature_name] = df_train[a] - df_train[b]
    df_test[diff_feature_name] = df_test[a] - df_test[b]
    feature_names.append(diff_feature_name)
    
    ratio_feature_name = f'ratio_{a}_{b}'
    df_train[ratio_feature_name] = df_train[a] / df_train[b]
    df_test[ratio_feature_name] = df_test[a] / df_test[b]
    feature_names.append(ratio_feature_name)<jupyter_output><empty_output><jupyter_text>Save features<jupyter_code>features_to_keep = feature_names
X_train = df_train[features_to_keep].values
X_test = df_test[features_to_keep].values
project.save_features(X_train, X_test, feature_names, feature_list_id)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09165460684997588, 'fraction_numerical': 0.000482392667631452, 'mean_word_length': 8.136563876651982, 'pattern_counts': {'":': 0, '<': 33, '<?xml version=': 0, '>': 33, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '659325', 'license': 'MIT', 'n_tokens_mistral': 695, 'n_tokens_neox': 683, 'n_words': 130, 'path': 'notebooks/unused/feature-meta-pairwise-interactions.ipynb', 'repo_name': 'MinuteswithMetrics/kaggle-quora-question-pairs', 'size': '2073'}