<jupyter_start><jupyter_text>Text to Text Explanation: Open Ended Text Generation Using GPT2 This notebook demonstrates how to get explanations for the output of gpt2 used for open ended text generation. In this demo, we use the pretrained gpt2 model provided by hugging face (https://huggingface.co/gpt2) to explain the generated text by gpt2. We further showcase how to get explanations for custom output generated text and plot global input token importances for any output generated token.<jupyter_code>import numpy as np
from transformers import AutoTokenizer, AutoModelForCausalLM
import shap
import torch<jupyter_output><empty_output><jupyter_text>Load model and tokenizer<jupyter_code>tokenizer = AutoTokenizer.from_pretrained("gpt2", use_fast=True)
model =  AutoModelForCausalLM.from_pretrained("gpt2").cuda()<jupyter_output><empty_output><jupyter_text>Below, we set certain model configurations. We need to define if the model is a decoder or encoder-decoder.This can be set through the 'is_decoder' or 'is_encoder_decoder' param in model's config file.We can also set custom model generation parameters which will be used during the output text generation decoding process.<jupyter_code># set model decoder to true and generation params
model.config.is_decoder=True
model.config.text_generation_params = {
    "do_sample": True,
    "max_length": 50,
    "temperature": 0.7,
    "top_k": 0
  }<jupyter_output><empty_output><jupyter_text>Define initial text<jupyter_code>s = ['I enjoy walking with my cute dog']<jupyter_output><empty_output><jupyter_text>Create an explainer object<jupyter_code>explainer = shap.Explainer(model,tokenizer)<jupyter_output>explainers.Partition is still in an alpha state, so use with caution...<jupyter_text>Compute shap values<jupyter_code>shap_values = explainer(s)<jupyter_output>Setting `pad_token_id` to 50256 (first `eos_token_id`) to generate sequence<jupyter_text>Visualize shap explanations<jupyter_code>shap.plots.text(shap_values)<jupyter_output><empty_output><jupyter_text>Another example...<jupyter_code>s=['Scientists confirmed the worst possible outcome: the massive asteroid will collide with Earth']
explainer = shap.Explainer(model,tokenizer)
shap_values = explainer(s)
shap.plots.text(shap_values)<jupyter_output><empty_output><jupyter_text>Custom text generation and debugging biased outputs Below we demonstrate the process of how to explain the liklihood of generating a particular output sentence given an input sentence using the model. For example, we ask a question:Which country's inhabitant (target) in the sentence "I know many people who are [target]." would have a high liklilhood of generating the token "vodka" in the output sentence "They love their vodka!" ?For this, we first define input-output sentence pairs<jupyter_code>biased_sentence_pairs = {
    "I know many people who are Russian." : "They love their vodka!",
    "I know many people who are Greek." : "They love their vodka!",
    "I know many people who are Australian." : "They love their vodka!",
    "I know many people who are American." : "They love their vodka!",
    "I know many people who are Italian." : "They love their vodka!",
    "I know many people who are Spanish." : "They love their vodka!",
    "I know many people who are German." : "They love their vodka!",
    "I know many people who are Indian." : "They love their vodka!"
}<jupyter_output><empty_output><jupyter_text>Next we define a custom text generation function which returns the target sentence ids of the output sentence of choice to be generated by the model.<jupyter_code>def generate_biased_output(x):
    # we get the target sentence by a dictionary lookup from the previously defined sentence pairs
    target_sentence = biased_sentence_pairs[x]
    target_sentence_ids = torch.tensor([tokenizer.encode(target_sentence)])
    return target_sentence_ids<jupyter_output><empty_output><jupyter_text>We wrap the model with a Teacher Forcing Logits scoring class and create a Text masker<jupyter_code>wrapped_model = shap.models.TeacherForcingLogits(model, tokenizer, generation_function_for_target_sentence_ids=generate_biased_output)
masker = shap.maskers.Text(tokenizer, mask_token = "...", collapse_mask_token=True)<jupyter_output><empty_output><jupyter_text>Create an explainer...<jupyter_code>explainer = shap.Explainer(wrapped_model,masker)<jupyter_output>explainers.Partition is still in an alpha state, so use with caution...<jupyter_text>Define the data to run the explainer on...<jupyter_code>s = list(biased_sentence_pairs.keys())<jupyter_output><empty_output><jupyter_text>Generate SHAP explanation values!<jupyter_code>shap_values = explainer(s)<jupyter_output>Partition explainer: 9it [00:10,  1.13s/it]<jupyter_text>Now that we have generated the SHAP values, we can have a look at the contribution of tokens in the input driving the token "vodka" in the output sentence using the text plot. Note: The red color indicates a positive contribution while the blue color indicates negative contribution and the intensity of the color shows its strength in the respective direction.<jupyter_code>shap.plots.text(shap_values)<jupyter_output><empty_output><jupyter_text>To view what input tokens impact (positively/negatively) the liklihood of generating the word "vodka", we plot the global token importances the word "vodka". Voila! Russians love their vodka, dont they? :)<jupyter_code>shap.plots.bar(shap_values[...,"vodka"])<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07835276967930029, 'fraction_numerical': 0.004555393586005831, 'mean_word_length': 6.417567567567567, 'pattern_counts': {'":': 4, '<': 61, '<?xml version=': 0, '>': 61, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '16', 'dir': 'jupyter-structured-clean-dedup', 'id': '163498', 'license': 'MIT', 'n_tokens_mistral': 1675, 'n_tokens_neox': 1591, 'n_words': 564, 'path': 'notebooks/text_examples/text_generation/Open Ended Text Generation Explanation Demo.ipynb', 'repo_name': 'text-shap/shap', 'size': '5488'}