<jupyter_start><jupyter_text>is_reg = False and is_multi = False This means that the last layer operation is Log_softmax(x) and m.crit is nll_lossIn case there are more than 2 classes (multi-class), we will change the out_sz to (N,C).To test this, we can simply add a random new class in the target variable, change the out size and run again. Please try this to understand it better!<jupyter_code>md = ColumnarModelData.from_data_frame('', val_idx, df, y.astype('int'), cat_flds=cat_vars, bs=512,
                                       is_reg=False,is_multi=False)
m = md.get_learner(emb_szs, len(df.columns)-len(cat_vars),
                   0.01, 2, [500,100], [0.01,0.1])
m.crit
# Verified that the m.crit is nll_loss and as per its definition
# in pytorch documentation we are providing the output size as (N,2)
m.lr_find()
m.sched.plot(1)
# Since we will be using the metrics = [accuracy] while calling the fit function, lets first observe the way it works
a = torch.rand(5,2)
#This emulates the class probabilities which is what we get with out_sz=2 and works with NLL loss

a
# This is what the accuracy function should grab and then compare to the actual target
# and find mean of all matching cases


torch.max(a, dim=1) 
# Let this be our hypothetical target

b = torch.LongTensor([0,0,1,1,1])
# This is how the code looks in the 'metrics.py' in fastai library

def accuracy(preds, targs):
    preds = torch.max(preds, dim=1)[1]
    return (preds==targs).float().mean()
# As expected, the acurracy is indeed the average of matching cases

accuracy(a,b)
m.fit(1e-5, n_cycle=2, cycle_len=1, cycle_mult=2, metrics=[accuracy])<jupyter_output><empty_output><jupyter_text>is_reg = False and is_multi = True This means that the last layer operation is Sigmoid(x) and m.crit is BCE<jupyter_code>md = ColumnarModelData.from_data_frame('', val_idx, df, y.astype(np.float32), cat_flds=cat_vars, bs=512,
                                       is_reg=False,is_multi=True)
m = md.get_learner(emb_szs, len(df.columns)-len(cat_vars),
                   0.01, 1, [500,100], [0.01,0.1])
m.crit
# Verified that the m.crit is BCE and as per its definition
# in pytorch documentationm we are providing the output size as (N,1)
m.lr_find()
m.sched.plot(1)
# Since we will be using the metrics = [accuracy_multi] while calling the fit function, lets first observe the way it works
a = torch.rand(5)
# This is what our probablities should look like, the assumption is that we are prediction the probability of class 1

a
# Filtering with threshold

(a>0.5).float()
# Let this be our hypothetical target

b = torch.FloatTensor([0,0,1,1,1])
def accuracy_multi(preds, targs, thresh=0.5):
    return ((preds>thresh).float()==targs).float().mean()
torch.eq((a>0.5).float(),b).float().mean() # With regular logic
accuracy_multi(a, b, thresh=0.5) # As per our function
m.fit(1e-5, n_cycle=2, cycle_len=1, cycle_mult=2, metrics=[accuracy_multi]) 
# I'm still not sure how to use the accuracy_multi if I don't set the default threshold in the function definition,
# I also can't pass just threshold during fit as it would require preds and targs


# Any help with this is very much appreciated !!!<jupyter_output><empty_output><jupyter_text>is_reg = True We are not working out this example here as it is clearly described in rossman notebook.<jupyter_code>This was a reply on the forum which really helped me understand the flow of the forward pass in the MixeInputModel

If not regression:
    If more than 1 class could be present for an observation:
        Use sigmoid
    Else only one class per observation:
        Use softmax

Else If regression and if a range is provided:
     Constrain the predictions be within the max and min of y_range

Otherwise, end with a linear activation<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.0923927349302448, 'fraction_numerical': 0.021058173203474598, 'mean_word_length': 4.142083897158322, 'pattern_counts': {'":': 0, '<': 13, '<?xml version=': 0, '>': 16, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '297979', 'license': 'MIT', 'n_tokens_mistral': 1259, 'n_tokens_neox': 1210, 'n_words': 473, 'path': 'StructuredLearnerExperiment.ipynb', 'repo_name': 'PranY/Passion-Projects', 'size': '3799'}