<jupyter_start><jupyter_text>Sentiment classification with transformers<jupyter_code># data processing tools
import os
import csv
import urllib.request
import pandas as pd
from tqdm import tqdm

# maths tools
import numpy as np
from scipy.special import softmax

# Huggingface tools
from transformers import AutoTokenizer
from transformers import TFAutoModelForSequenceClassification
from transformers import pipeline, set_seed<jupyter_output><empty_output><jupyter_text>Choose task specific model The ```twitter-roberta-base``` model has been finetuned on a number of slightly different but related sentiment-style tasks. Specific models are finetuned to predict the following labels:- emotion- hate- irony- offensive- sentiment- emoji<jupyter_code>task='sentiment'
MODEL = f"cardiffnlp/twitter-roberta-base-{task}"<jupyter_output><empty_output><jupyter_text>Initialize tokenizer We initalize a pretrained tokenizer, which we need to use to tokenize our texts.<jupyter_code>tokenizer = AutoTokenizer.from_pretrained(MODEL)<jupyter_output><empty_output><jupyter_text>Download label mappings This specific model requires us to download the labels that we're going to use separately - this isn't the case with every model. We'll see more on that below, under ```Pipelines```.<jupyter_code># download label mapping
labels=[]
mapping_link = f"https://raw.githubusercontent.com/cardiffnlp/tweeteval/main/datasets/{task}/mapping.txt"
with urllib.request.urlopen(mapping_link) as f:
    html = f.read().decode('utf-8').split("\n")
    csvreader = csv.reader(html, delimiter='\t')
labels = [row[1] for row in csvreader if len(row) > 1]<jupyter_output><empty_output><jupyter_text>Initialize model<jupyter_code># # TF
model = TFAutoModelForSequenceClassification.from_pretrained(MODEL)
#model.save_pretrained(MODEL)<jupyter_output><empty_output><jupyter_text>Predict text<jupyter_code># text input    
text = "Oh, well, that sounds just great"

# encode using tokenizer
encoded_input = tokenizer(text, return_tensors='tf')

# get output
output = model(encoded_input)

# get outputs as numpy array
scores = output[0][0].numpy()

# perform softmax classification
scores = softmax(scores)<jupyter_output><empty_output><jupyter_text>Get ranked scores<jupyter_code>ranking = np.argsort(scores)
ranking = ranking[::-1]
for i in range(scores.shape[0]):
    l = labels[ranking[i]]
    s = scores[ranking[i]]
    print(f"{i+1}) {l} {np.round(float(s), 4)}")<jupyter_output><empty_output><jupyter_text>HuggingFace pipelines Pipelines give us less control over the fine details but instead allow us to quickly generate results with default parameters.<jupyter_code>classifier = pipeline("text-classification", 
                      model = "cardiffnlp/twitter-roberta-base-emotion", 
                      return_all_scores=True)
scores = classifier("you suck! 🤬")
scores<jupyter_output><empty_output><jupyter_text>These results don't look great because of the problem with the labels that we saw earlier. We can obviously fix that afterwards by mapping the label names onto these results. Alternatively, we could use a different model that doesn't have these problems.<jupyter_code>classifier = pipeline("text-classification",
                      model='bhadresh-savani/distilbert-base-uncased-emotion', 
                      return_all_scores=True)
prediction = classifier("you suck! 🤬")
all_scores = []
for idx, row in tqdm(data.iterrows()):
    prediction = classifier(row["text"])
    all_scores.append((row["text"], prediction))
classifier = pipeline("sentiment-analysis")
classifier("I loved Star Wars so much!")
prediction<jupyter_output><empty_output><jupyter_text>Score dataframe<jupyter_code>filename = os.path.join("..","..","CDS-LANG", "reviews", "yelp_labelled.txt")

data = pd.read_table(filename, header=None, names=["text", "label"])<jupyter_output><empty_output><jupyter_text>Text generation<jupyter_code>generator = pipeline('text-generation', model='gpt2')
set_seed(42)
generator("Hello, I'm a language model,", 
          max_length=50, 
          num_return_sequences=5)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1079171741778319, 'fraction_numerical': 0.0036540803897685747, 'mean_word_length': 5.959322033898305, 'pattern_counts': {'":': 0, '<': 45, '<?xml version=': 0, '>': 46, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '551367', 'license': 'MIT', 'n_tokens_mistral': 1321, 'n_tokens_neox': 1256, 'n_words': 325, 'path': 'notebooks/session11_inclass_rdkm.ipynb', 'repo_name': 'agnesbn/cds-language', 'size': '4105'}