<jupyter_start><jupyter_text>Text model explanation using Lime Counterfactuals  Imports and installs<jupyter_code>import os
import random
import spacy

import numpy                 as np
import tensorflow            as tf
import tensorflow_hub        as hub
import tensorflow_datasets   as tfds
random.seed(0)
np.random.seed(0)
tf.random.set_seed(0)
!pip install spacy lime pydictionary
!python -m spacy download en_core_web_lg
os.environ["TFHUB_MODEL_LOAD_FORMAT"] = "COMPRESSED"
print("Version: ", tf.__version__)
print("Eager mode: ", tf.executing_eagerly())
print("Hub version: ", hub.__version__)
print("GPU is", "available" if tf.config.experimental.list_physical_devices("GPU") else "NOT AVAILABLE")
!rm -rf explainable_ai
!git clone https://github.com/kartikparnami/explainable_ai.git
from explainable_ai.counterfactual.lc_text import LimeCounterfactualText<jupyter_output>Cloning into 'explainable_ai'...
remote: Enumerating objects: 33, done.[K
remote: Counting objects: 100% (33/33), done.[K
remote: Compressing objects: 100% (27/27), done.[K
remote: Total 33 (delta 2), reused 33 (delta 2), pack-reused 0[K
Unpacking objects: 100% (33/33), done.<jupyter_text>Construct model and utilities<jupyter_code>def mask_to_categorical(data, mask):
    mask = tf.one_hot(tf.cast(mask, tf.int32), 2)
    mask = tf.cast(mask, tf.float32)
    return data, mask

# Split the training set into 60% and 40%, so we'll end up with 15,000 examples
# for training, 10,000 examples for validation and 25,000 examples for testing.
train_data, validation_data, test_data = tfds.load(
    name="yelp_polarity_reviews", 
    split=('train[:60%]', 'train[60%:]', 'test'),
    as_supervised=True)

train_examples_batch, train_labels_batch = next(iter(train_data.batch(10).map(mask_to_categorical)))
embedding = "https://tfhub.dev/google/nnlm-en-dim50/2"
hub_layer = hub.KerasLayer(embedding, input_shape=[], 
                           dtype=tf.string, trainable=True)
hub_layer(train_examples_batch[:3])

model = tf.keras.Sequential()
model.add(hub_layer)
model.add(tf.keras.layers.Dense(128, activation='relu'))
model.add(tf.keras.layers.Dense(64, activation='relu'))
model.add(tf.keras.layers.Dense(32, activation='relu'))
model.add(tf.keras.layers.Dense(16, activation='relu'))
model.add(tf.keras.layers.Dense(2, activation='softmax'))

model.compile(optimizer='adam',
              loss=tf.keras.losses.BinaryCrossentropy(from_logits=False),
              metrics=['accuracy'])

history = model.fit(train_data.shuffle(10000).batch(512).map(mask_to_categorical),
                    epochs=5,
                    validation_data=validation_data.batch(512).map(mask_to_categorical),
                    verbose=1)
def predict_lr_counterfactual(texts):
    return model.predict(texts)<jupyter_output><empty_output><jupyter_text>Lime Counterfactual text explanation and visualization  Real positive examples<jupyter_code>lc_explainer = LimeCounterfactualText()
######################
# Correct Prediction #
######################

text = "Cheap and delicious! I eat here about once a week because it fills me up for cheap price and the people who work there are really fun. it\'s like a subway sandwich restaurant for Mexican food. I love the burrito w/ Cheese, Beans & rice w/ sour cream, less than $5 and best in town for that price."
lc_explainer.explain_instance(text, predict_lr_counterfactual)<jupyter_output>/usr/local/lib/python3.6/dist-packages/lime/lime_text.py:114: FutureWarning: split() requires a non-empty pattern match.
  self.as_list = [s for s in splitter.split(self.raw) if s]
/usr/local/lib/python3.6/dist-packages/lime/lime_text.py:114: FutureWarning: split() requires a non-empty pattern match.
  self.as_list = [s for s in splitter.split(self.raw) if s]
/usr/local/lib/python3.6/dist-packages/lime/lime_text.py:114: FutureWarning: split() requires a non-empty pattern match.
  self.as_list = [s for s in splitter.split(self.raw) if s]
/usr/local/lib/python3.6/dist-packages/lime/lime_text.py:114: FutureWarning: split() requires a non-empty pattern match.
  self.as_list = [s for s in splitter.split(self.raw) if s]
/usr/local/lib/python3.6/dist-packages/lime/lime_text.py:114: FutureWarning: split() requires a non-empty pattern match.
  self.as_list = [s for s in splitter.split(self.raw) if s]
/usr/local/lib/python3.6/dist-packages/lime/lime_text.py:114: FutureWarning: split() requires a[...]<jupyter_text>The words with the most contribution towards the positive prediction are replaced one-by-one by the baseline to understand their importance to the prediction. Lime is used as the heuristic to get their relative importance.<jupyter_code>######################
# Correct Prediction #
######################

text = "Cheap and delicious! I eat here about once a week because it fills me up for cheap price and the people who work there are really fun. it\'s like a subway sandwich restaurant for Mexican food. I love the burrito w/ Cheese, Beans & rice w/ sour cream, less than $5 and best in town for that price."
lc_explainer.explain_instance(text, predict_lr_counterfactual, with_antonyms=True)<jupyter_output>/usr/local/lib/python3.6/dist-packages/lime/lime_text.py:114: FutureWarning: split() requires a non-empty pattern match.
  self.as_list = [s for s in splitter.split(self.raw) if s]
/usr/local/lib/python3.6/dist-packages/lime/lime_text.py:114: FutureWarning: split() requires a non-empty pattern match.
  self.as_list = [s for s in splitter.split(self.raw) if s]
/usr/local/lib/python3.6/dist-packages/lime/lime_text.py:114: FutureWarning: split() requires a non-empty pattern match.
  self.as_list = [s for s in splitter.split(self.raw) if s]
/usr/local/lib/python3.6/dist-packages/lime/lime_text.py:114: FutureWarning: split() requires a non-empty pattern match.
  self.as_list = [s for s in splitter.split(self.raw) if s]
/usr/local/lib/python3.6/dist-packages/lime/lime_text.py:114: FutureWarning: split() requires a non-empty pattern match.
  self.as_list = [s for s in splitter.split(self.raw) if s]
/usr/local/lib/python3.6/dist-packages/lime/lime_text.py:114: FutureWarning: split() requires a[...]<jupyter_text>The words with the most contribution towards the positive prediction are replaced one-by-one by an antonym to understand their importance to the prediction. Lime is used as the heuristic to get their relative importance.<jupyter_code>########################
# Incorrect Prediction #
########################

text = "I hate this place. I hate it because I could be checking out a different restaurant but there I am, always coming back for those freaking nachos! Ok, next time i will try and skip Margaritaville but I won't make any promises!"
lc_explainer.explain_instance(text, predict_lr_counterfactual)<jupyter_output>/usr/local/lib/python3.6/dist-packages/lime/lime_text.py:114: FutureWarning: split() requires a non-empty pattern match.
  self.as_list = [s for s in splitter.split(self.raw) if s]
/usr/local/lib/python3.6/dist-packages/lime/lime_text.py:114: FutureWarning: split() requires a non-empty pattern match.
  self.as_list = [s for s in splitter.split(self.raw) if s]
/usr/local/lib/python3.6/dist-packages/lime/lime_text.py:114: FutureWarning: split() requires a non-empty pattern match.
  self.as_list = [s for s in splitter.split(self.raw) if s]
/usr/local/lib/python3.6/dist-packages/lime/lime_text.py:114: FutureWarning: split() requires a non-empty pattern match.
  self.as_list = [s for s in splitter.split(self.raw) if s]
/usr/local/lib/python3.6/dist-packages/lime/lime_text.py:114: FutureWarning: split() requires a non-empty pattern match.
  self.as_list = [s for s in splitter.split(self.raw) if s]
/usr/local/lib/python3.6/dist-packages/lime/lime_text.py:114: FutureWarning: split() requires a[...]<jupyter_text>The words with the most contribution towards the negative prediction are replaced one-by-one by the baseline to understand their importance to the prediction even though the real label is positive. Lime is used as the heuristic to get their relative importance.<jupyter_code>########################
# Incorrect Prediction #
########################

text = "I hate this place. I hate it because I could be checking out a different restaurant but there I am, always coming back for those freaking nachos! Ok, next time i will try and skip Margaritaville but I won't make any promises!"
lc_explainer.explain_instance(text, predict_lr_counterfactual, with_antonyms=True)<jupyter_output>/usr/local/lib/python3.6/dist-packages/lime/lime_text.py:114: FutureWarning: split() requires a non-empty pattern match.
  self.as_list = [s for s in splitter.split(self.raw) if s]
/usr/local/lib/python3.6/dist-packages/lime/lime_text.py:114: FutureWarning: split() requires a non-empty pattern match.
  self.as_list = [s for s in splitter.split(self.raw) if s]
/usr/local/lib/python3.6/dist-packages/lime/lime_text.py:114: FutureWarning: split() requires a non-empty pattern match.
  self.as_list = [s for s in splitter.split(self.raw) if s]
/usr/local/lib/python3.6/dist-packages/lime/lime_text.py:114: FutureWarning: split() requires a non-empty pattern match.
  self.as_list = [s for s in splitter.split(self.raw) if s]<jupyter_text>The words with the most contribution towards the negative prediction are replaced one-by-one by an antonym to understand their importance to the prediction even though the real label is positive. Lime is used as the heuristic to get their relative importance.  Real negative examples<jupyter_code>lc_explainer = LimeCounterfactualText()
######################
# Correct Prediction #
######################

text = "Perhaps it was an off night, but we were really disappointed in our take-out. My husband's pork fried rice was not fried rice...still don't know what it was. His Gen. Tso's chicken was flat and my cashew chicken was flavorless. In the past it has been good food."
lc_explainer.explain_instance(text, predict_lr_counterfactual)<jupyter_output>/usr/local/lib/python3.6/dist-packages/lime/lime_text.py:114: FutureWarning: split() requires a non-empty pattern match.
  self.as_list = [s for s in splitter.split(self.raw) if s]
/usr/local/lib/python3.6/dist-packages/lime/lime_text.py:114: FutureWarning: split() requires a non-empty pattern match.
  self.as_list = [s for s in splitter.split(self.raw) if s]
/usr/local/lib/python3.6/dist-packages/lime/lime_text.py:114: FutureWarning: split() requires a non-empty pattern match.
  self.as_list = [s for s in splitter.split(self.raw) if s]
/usr/local/lib/python3.6/dist-packages/lime/lime_text.py:114: FutureWarning: split() requires a non-empty pattern match.
  self.as_list = [s for s in splitter.split(self.raw) if s]
/usr/local/lib/python3.6/dist-packages/lime/lime_text.py:114: FutureWarning: split() requires a non-empty pattern match.
  self.as_list = [s for s in splitter.split(self.raw) if s]
/usr/local/lib/python3.6/dist-packages/lime/lime_text.py:114: FutureWarning: split() requires a[...]<jupyter_text>The words with the most contribution towards the negative prediction are replaced one-by-one by the baseline to understand their importance to the prediction. Lime is used as the heuristic to get their relative importance.<jupyter_code>######################
# Correct Prediction #
######################

text = "Perhaps it was an off night, but we were really disappointed in our take-out. My husband's pork fried rice was not fried rice...still don't know what it was. His Gen. Tso's chicken was flat and my cashew chicken was flavorless. In the past it has been good food."
lc_explainer.explain_instance(text, predict_lr_counterfactual, with_antonyms=True)<jupyter_output>/usr/local/lib/python3.6/dist-packages/lime/lime_text.py:114: FutureWarning: split() requires a non-empty pattern match.
  self.as_list = [s for s in splitter.split(self.raw) if s]
/usr/local/lib/python3.6/dist-packages/lime/lime_text.py:114: FutureWarning: split() requires a non-empty pattern match.
  self.as_list = [s for s in splitter.split(self.raw) if s]
/usr/local/lib/python3.6/dist-packages/lime/lime_text.py:114: FutureWarning: split() requires a non-empty pattern match.
  self.as_list = [s for s in splitter.split(self.raw) if s]
/usr/local/lib/python3.6/dist-packages/lime/lime_text.py:114: FutureWarning: split() requires a non-empty pattern match.
  self.as_list = [s for s in splitter.split(self.raw) if s]<jupyter_text>The words with the most contribution towards the negative prediction are replaced one-by-one by an antonym to understand their importance to the prediction. Lime is used as the heuristic to get their relative importance.<jupyter_code>########################
# Incorrect Prediction #
########################

text = "On the southwest corner of Sandy Porter and Tryon, you'll find this Food Lion in the strip mall. I've not shopped at many FL's, so I can't tell you if this is bigger or smaller than average. What I can tell you is that the prices here seem higher than Harris-Teeter, Bi-Lo, Wal-Mart and Super Target. While they have everything you need for a full grocery shop, including frozen, dairy, produce, beer, wine and meats, it's not going to be cheap.\\n\\nIf you do decide to shop with the Lion on a regular basis, pick up one of their MVP shopper loyalty cards - this will save you a good bit."
lc_explainer.explain_instance(text, predict_lr_counterfactual)<jupyter_output>/usr/local/lib/python3.6/dist-packages/lime/lime_text.py:114: FutureWarning: split() requires a non-empty pattern match.
  self.as_list = [s for s in splitter.split(self.raw) if s]
/usr/local/lib/python3.6/dist-packages/lime/lime_text.py:114: FutureWarning: split() requires a non-empty pattern match.
  self.as_list = [s for s in splitter.split(self.raw) if s]
/usr/local/lib/python3.6/dist-packages/lime/lime_text.py:114: FutureWarning: split() requires a non-empty pattern match.
  self.as_list = [s for s in splitter.split(self.raw) if s]
/usr/local/lib/python3.6/dist-packages/lime/lime_text.py:114: FutureWarning: split() requires a non-empty pattern match.
  self.as_list = [s for s in splitter.split(self.raw) if s]
/usr/local/lib/python3.6/dist-packages/lime/lime_text.py:114: FutureWarning: split() requires a non-empty pattern match.
  self.as_list = [s for s in splitter.split(self.raw) if s]
/usr/local/lib/python3.6/dist-packages/lime/lime_text.py:114: FutureWarning: split() requires a[...]<jupyter_text>The words with the most contribution towards the positive prediction are replaced one-by-one by the baseline to understand their importance to the prediction even though the real label is negative. Lime is used as the heuristic to get their relative importance.<jupyter_code>########################
# Incorrect Prediction #
########################

text = "On the southwest corner of Sandy Porter and Tryon, you'll find this Food Lion in the strip mall. I've not shopped at many FL's, so I can't tell you if this is bigger or smaller than average. What I can tell you is that the prices here seem higher than Harris-Teeter, Bi-Lo, Wal-Mart and Super Target. While they have everything you need for a full grocery shop, including frozen, dairy, produce, beer, wine and meats, it's not going to be cheap.\\n\\nIf you do decide to shop with the Lion on a regular basis, pick up one of their MVP shopper loyalty cards - this will save you a good bit."
lc_explainer.explain_instance(text, predict_lr_counterfactual, with_antonyms=True)<jupyter_output>/usr/local/lib/python3.6/dist-packages/lime/lime_text.py:114: FutureWarning: split() requires a non-empty pattern match.
  self.as_list = [s for s in splitter.split(self.raw) if s]
/usr/local/lib/python3.6/dist-packages/lime/lime_text.py:114: FutureWarning: split() requires a non-empty pattern match.
  self.as_list = [s for s in splitter.split(self.raw) if s]
/usr/local/lib/python3.6/dist-packages/lime/lime_text.py:114: FutureWarning: split() requires a non-empty pattern match.
  self.as_list = [s for s in splitter.split(self.raw) if s]
/usr/local/lib/python3.6/dist-packages/lime/lime_text.py:114: FutureWarning: split() requires a non-empty pattern match.
  self.as_list = [s for s in splitter.split(self.raw) if s]
/usr/local/lib/python3.6/dist-packages/lime/lime_text.py:114: FutureWarning: split() requires a non-empty pattern match.
  self.as_list = [s for s in splitter.split(self.raw) if s]
/usr/local/lib/python3.6/dist-packages/lime/lime_text.py:114: FutureWarning: split() requires a[...]
{'fraction_non_alphanumeric': 0.1263969873663751, 'fraction_numerical': 0.018950437317784258, 'mean_word_length': 4.803665844201621, 'pattern_counts': {'":': 0, '<': 32, '<?xml version=': 0, '>': 32, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '305540', 'license': 'MIT', 'n_tokens_mistral': 5497, 'n_tokens_neox': 5246, 'n_words': 1643, 'path': 'examples/counterfactual/ExplainableAI_LimeCounterfactural_Text.ipynb', 'repo_name': 'kartikparnami/explainable_ai', 'size': '16464'}