<jupyter_start><jupyter_text>Intro to Ecco -- Making Language Models More Transparent
This notebook is an intro to [Ecco](https://www.eccox.io) and is the companion to the video: [Take A Look Inside Language Models With Ecco | PyData Khobar](https://www.youtube.com/watch?v=rHrItfNeuh0).<jupyter_code># Install Ecco. This assumes you have pytorch installed.
!pip install ecco
import warnings
warnings.filterwarnings('ignore')
import ecco

# Load pre-trained language model. Setting 'activations' to True tells Ecco to capture neuron activations.
# distillgpt is a distilled GPT2 model. You can also try 'gpt2' 
lm = ecco.from_pretrained('distilgpt2', activations=True)<jupyter_output><empty_output><jupyter_text>OverviewComplete the sentence:`It was a matter of ____`<jupyter_code>text = " it was a matter of"

# Generate one token
output_1 = lm.generate(text, generate=1, do_sample=False)
# Show the top 10 candidate output tokens for position #5. 
# Layer 5 is the last layer in the model.
output_1.layer_predictions(position=5, layer=5, topk=10)<jupyter_output>Predictions for position 5<jupyter_text>This view shows the top 10 candidate tokens, their probability, and their rankings.

 Comparing two token candidates for a single position<jupyter_code># What are the token IDs of the two words?
lm.tokenizer(" principle principal")
# Compare the rankings of "Principle" and "Principal" across layers
output_1.rankings_watch(watch=[7989, 10033], position=5)<jupyter_output><empty_output><jupyter_text>Exploring World Knowledge And Layer AnalysisDoes the model "know" where Heathrow airport is located? To probe the model, we let's try the input sentence: `Heathrow airport is located in ____`<jupyter_code>text = " Heathrow airport is located in"
output_2 = lm.generate(text, generate=5, do_sample=False)<jupyter_output><empty_output><jupyter_text>While the output is not incorrect, it doesn't really anwer the question we are after. Let's slightly change the input sentence:<jupyter_code>text = " Heathrow airport is located in the city of"
output_2 = lm.generate(text, generate=1, do_sample=False)
# What other tokens were possible to output in place of "London"?
output_2.layer_predictions(position=9, layer=5, topk=30)
# Now that the model has selcted the tokens "London . \n"
# How did each layer rank these tokens during processing?
output_2.rankings()<jupyter_output><empty_output><jupyter_text>This visualization is based on the great visual treatment by nostalgebraist in [Interpreting GPT: the logit lens](https://www.lesswrong.com/posts/AcKRB8wDpdaN6v6ru/interpreting-gpt-the-logit-lens).  Probing the model's world knowledge What happens if we present the following input sentence to the model:`The countries of the European Union are:\n1. Austria\n2. Belgium\n3. Bulgaria\n4. ___________`Namely, we have these questions:* Q. Will the model continue the numbering correctly?* Q. Will it succeed in following the formatting?* Q. Will it succeed in naming countries? European ones?* Q. Will the model "notice" the alphabetical order of the list? Will it follow it?<jupyter_code>text= "The countries of the European Union are:\n1. Austria\n2. Belgium\n3. Bulgaria\n4."

output_3 = lm.generate(text, generate=20, do_sample=True)<jupyter_output><empty_output><jupyter_text>The `rankings()` visualization view shows us at which layers the model resolved the output token for each position.<jupyter_code>output_3.rankings()<jupyter_output><empty_output><jupyter_text>The `saliency()` visualization shows which tokens contributed the most towards generating each output token (using the gradient X Input method):<jupyter_code>output_3.saliency()<jupyter_output><empty_output><jupyter_text>Detailed saliency view
We can see a more detailed view of the saliency values using the detailed view:<jupyter_code>output_3.saliency(style="detailed")<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08442728442728442, 'fraction_numerical': 0.014414414414414415, 'mean_word_length': 6.304511278195489, 'pattern_counts': {'":': 0, '<': 36, '<?xml version=': 0, '>': 36, 'https://': 3, 'lorem ipsum': 0, 'www.': 3, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '103563', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 1232, 'n_tokens_neox': 1164, 'n_words': 422, 'path': 'notebooks/Language_Models_and_Ecco_PyData_Khobar.ipynb', 'repo_name': 'Haziqa-coder/ecco', 'size': '3885'}