<jupyter_start><jupyter_text>Training symboloc regression modelsThis notebook illustrates how to use  scripts developed to train symbolic regression models. For more information refer to the [main manuscript](xxx)  The following examples were run with the following dependencies:* numpy == 1.20.0* pandas == 1.2.1* sympy == 1.9* scikit-learn == 1.0.1* autofeat == 2.0.10  If you have unexpected errors you might try to reproduce this environment.<jupyter_code>import numpy as np
import pandas as pd
import workflows as wf<jupyter_output><empty_output><jupyter_text>Training dataFor this example we generate artifical training data from a arbitrarily chosen expression. In your application, however, you will replace this function for a function opening an existing dataset from, e.g. a pickle file, or a csv. Your training data should be in a pandas dataframe of shape (n samples) x (m predictors + target).<jupyter_code># -------------- TRAINING DATA -------------------

def generate_training_data():
    """Example function to generate toy training set"""

    npoints = 50
    t_coords = np.linspace(243, 330, npoints) #temperature samples
    c_coords = np.linspace(0.1, 3, npoints) #concentration samples
    b1, b2 = 1, -2e-3 #coefficients

    #training dataframe
    train_data = pd.DataFrame(data = np.array([np.repeat(t_coords,npoints**2), np.hstack((npoints**2)*(c_coords,))]).T, 
                            columns=['T', 'c'])

    #apply expression to be found via symbolic regression: k = b1*c*t**1.5 + b2*t*c**1.5
    train_data['k'] = b1*train_data['c']*train_data['T'].pow(0.5) + b2*train_data['T']*train_data['c'].pow(1.5)
    
    return train_data<jupyter_output><empty_output><jupyter_text>Setting hyperparametersThe workflow can be conceptually divided into:* A feature generation step, where hundreds of features are generated from the initial predictors by combining these using non-linear transformations.* A feature selection step, where  most candidate features are filtered out due to multiple criteria, e.g. close-to-zero coefrficients, non-sensical dimensions, poor correlation to target, high-correlation to simpler features, etc.More details about the steps consult the [Autofeat documentation](https://github.com/cod3licious/autofeat).We first set all hyperparamters, for the feature generation and selection steps.<jupyter_code># --------------  HYPER-PARAMETERS FEATURE GENERATION -------------------

#how to scale data. Supported 'standard_nomean', 'standard', 'none'
SCALING_TYPE = 'standard_nomean' 

#whether to leave intercept to vary freely (True) or constrain its value to y0 = 0 (False).
FIT_INTERCEPT = False

# Autofeat hyperparameter.
# number of times features are combined to obtain ever more complex features.
# example FEATENG_STEPS = 3 with sqrt transformations will find terms like sqrt(sqrt(sqrt(x)))
FEATENG_STEPS = 5

# Autofeat hyperparameter.
# Units of predictors. Keys must match column names in dataframe. 
# Ignored predictors are assumed to be dimensionless.
UNITS = {"T": "1/K",
        "c": "mol/kg"}

# Autofeat hyperparameter.
# Number of iterations for filtering out generated features.
FEATSEL_RUNS = 3

# Autofeat hyperparameter.
# Set of non-linear transformations to be applied to initial predictors.
# Autofeat throws an error when using a single transformation. 
# Repeat your transformation as a workaround if you only want o use one.
TRANSFORMATIONS = ["sqrt", "sqrt"]


# --------------  HYPER-PARAMETERS FEATURE SELECTION -------------------

# n-standard deviations criterion to choose optimal alpha from Cross Validation. 
# Higher STD_ALPHA lead to sparser solutions.
STD_ALPHA = 1 

#t-statistic rejection threshold. Coefficients with t-statistic < REJECTION_THR are rejected.
REJECTION_THR = 2<jupyter_output><empty_output><jupyter_text>Run workflowsWorkflows are essentially pre-defined recipes of training steps:* standardization* feature generation* determination of coecfficients and t-values* feature selection* final training* generate trained model objectsBelow we instantiate a workflow object, with the required parameters and data. Then we run the workflow method that execute the steps in the recipe.<jupyter_code># ---------------- INSTANTIATE WORKFLOW --------------------------------------
training_data = generate_training_data()

workflow = wf.WorkflowAF(feateng_steps = FEATENG_STEPS,
                        units =  UNITS,
                        featsel_runs = FEATSEL_RUNS,
                        transformations = TRANSFORMATIONS,
                        xtrain = training_data[['c','T']], 
                        ytrain = training_data['k'], 
                        scaling_type = SCALING_TYPE,
                        stdalpha =  STD_ALPHA, 
                        rejection_thresshold = REJECTION_THR, 
                        fit_intercept = FIT_INTERCEPT) 

# ---------------- RUN TRAINING WORKFLOW --------------------------------------
trained_workflow = workflow.run_workflow()<jupyter_output>[AutoFeat] Warning: This just calls fit_transform() but does not return the transformed dataframe.
[AutoFeat] It is much more efficient to call fit_transform() instead of fit() and transform()!
[AutoFeat] Applying the Pi Theorem
[AutoFeat] The 5 step feature engineering process could generate up to 1354 features.
[AutoFeat] With 125000 data points this new feature matrix would use about 0.68 gb of space.
[feateng] Step 1: transformation of original features
[feateng] Generated 2 transformed features from 2 original features - done.
[feateng] Step 2: first combination of features
[feateng] Generated 6 feature combinations from 6 original feature tuples - done.
[feateng] Step 3: transformation of new features
[feateng] Generated 6 transformed features from 6 original features - done.
[feateng] Step 4: combining old and new features
[feateng] Generated 36 feature combinations from 48 original feature tuples - done.
[feateng] Step 5: combining new features
[feateng] Generated 58 feature co[...]<jupyter_text>The trained model, its predictions and evaluationThe result of the run_workflow() method is a TrainedWorkflow object, which stores the resulting coefficients after training, the expression found and has a handy method predict() to apply the model to new data. We use the predict method to compare the real target with the predction from the model, and compute regression metrics.<jupyter_code># ------------------------- INSPECT SYMBOLIC MODEL --------------------------
trained_workflow.coeff_table
trained_workflow.eqn
# ------------------------- EVALUATE SYMBOLIC MODEL --------------------------
import sklearn.metrics as skmetrics 

y_real = training_data['k']
y_hat = trained_workflow.predict(x = training_data[['c','T']])

print('mse: {}'.format(skmetrics.mean_squared_error(y_real, y_hat)))
print('r2: {}'.format(skmetrics.r2_score(y_real, y_hat)))
print('mape: {}'.format(skmetrics.mean_absolute_percentage_error(y_real, y_hat)))
training_data['k'].describe()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1200739792289088, 'fraction_numerical': 0.012519561815336464, 'mean_word_length': 3.541343669250646, 'pattern_counts': {'":': 2, '<': 21, '<?xml version=': 0, '>': 20, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '640607', 'license': 'MIT', 'n_tokens_mistral': 2034, 'n_tokens_neox': 1853, 'n_words': 717, 'path': 'example_use.ipynb', 'repo_name': 'BIG-MAP/SR-electrolytes', 'size': '7029'}