<jupyter_start><jupyter_text>Set-upThis set-up assumes that the working directory (`os.curdir`) is where the notebook is.<jupyter_code>import os
import sys
this_notebook_dir = os.curdir
project_root_dir = os.path.relpath(os.path.join('..', '..'), this_notebook_dir)
if project_root_dir not in sys.path:
    sys.path += [project_root_dir]
from pprint import pprint<jupyter_output><empty_output><jupyter_text>Loading data and modelWe will now a dataset<jupyter_code>from src.data.dataload import *
data = load_sst()
print(f'loaded dataset {data.NAME}')
train, dev, test = data.train_val_test<jupyter_output>loaded dataset sst<jupyter_text>Loading a model for the dataset<jupyter_code>from src.models.bcnmodel import *
from src.models.bertmodel import *
model = BCNModel()
print(f'expecting location for the model file at '
      f'"{model._get_model_filepath_for_dataset(data)}"')
model.load_model(data)
print(f'loaded model {model} of type {model.MODELTYPE} for {data.NAME}')<jupyter_output>ignoring warnings from spacy
expecting location for the model file at "../../models/bcn-sst_output/model.tar.gz"
loaded model <src.models.bcnmodel.BCNModel object at 0x7faca645dee0> of type allennlp for sst<jupyter_text>ExplainersCreating an explainer for the model<jupyter_code>from src.explainers.explainers import *
explainer = LimeExplainer(model, num_samples=2000)
print(f'using explainer {type(explainer)} with model {explainer.model} and dataset {explainer.model.dataset_finetune.NAME}')<jupyter_output>using explainer <class 'src.explainers.explainers.LimeExplainer'> with model <src.models.bcnmodel.BCNModel object at 0x7faca645dee0> and dataset sst<jupyter_text>Run explainer<jupyter_code>inds = np.arange(5, 10)
X = explainer.explain_instances(dev.sentence[inds])
print('SENTENCE:', dev.sentence[inds[0]])
tokenized = model.tokenizer.tokenize(dev.sentence[inds[0]])
if type(explainer) == LimeExplainer:
    scores, pred, inds, tokens = X
    print('tokens', [tokenized[t] for t in tokens[0]])
    print('scores', ['%.3f' % s for s in scores[0]])
elif type(explainer) == SHAPExplainer:
    shap_values = X
    print('shap values', shap_values[0])
elif type(explainer) == AllenNLPExplainer:
    grads, labels = X
    print(tokenized)
    print('gradients', ['%.3f' % g for g in grads[0]])<jupyter_output>SENTENCE: Half Submarine flick , Half Ghost Story , All in one criminally neglected film
tokens [in, All, Ghost, Half, Half, flick, ,, one, Submarine, ,, Story]
scores ['0.130', '0.053', '0.039', '0.026', '0.020', '-0.002', '-0.002', '-0.003', '-0.004', '-0.005', '-0.009']
{'fraction_non_alphanumeric': 0.13548136645962733, 'fraction_numerical': 0.027562111801242236, 'mean_word_length': 5.983739837398374, 'pattern_counts': {'":': 0, '<': 20, '<?xml version=': 0, '>': 20, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '333605', 'license': 'MIT', 'n_tokens_mistral': 965, 'n_tokens_neox': 921, 'n_words': 220, 'path': 'notebooks/code-demo/04-explainers.ipynb', 'repo_name': 'stevieg3/ucl-nlp-group-project', 'size': '2576'}