<jupyter_start><jupyter_text>BERT Based Sentiment Analysis Model ServerThe model used here, was trained with the concept of transfer learning  i.e. taking huggingface transformers pretrained BERT model and further training it on a custom dataset of reviews. this yields a sentiment analysis model based on the prior knowledge of BERT. The model server is given a list of texts and outputs a list of labels corresponding to its prediction.The labels express the sentiment of the writer towards the topic of the text:0 for negative sentiment, 1 for neutral and 2 for positive.The model file (~430 MB), can be downloaded to your local environment from: https://iguazio-sample-data.s3.amazonaws.com/models/model.pt<jupyter_code># nuclio: ignore
import nuclio
%nuclio config kind="nuclio:serving"
%nuclio config spec.build.baseImage="mlrun/ml-models-gpu"
%nuclio config spec.readiness_timeout = 500
%nuclio env MODEL_CLASS=SentimentClassifierServing<jupyter_output>%nuclio: setting kind to 'nuclio:serving'
%nuclio: setting spec.build.baseImage to 'mlrun/ml-models-gpu'
%nuclio: setting spec.readiness_timeout to 500
%nuclio: setting 'MODEL_CLASS' environment variable<jupyter_text>function code below is the model architecture, implemented with pytorch whose main component is bert.<jupyter_code>import torch
import torch.nn as nn
from transformers import BertModel, BertTokenizer
from cloudpickle import dumps

PRETRAINED_MODEL = 'bert-base-cased'
tokenizer = BertTokenizer.from_pretrained('bert-base-cased')

class BertSentimentClassifier(nn.Module):
    def __init__(self, n_classes):
        super(BertSentimentClassifier, self).__init__()
        self.bert = BertModel.from_pretrained(PRETRAINED_MODEL)
        self.dropout = nn.Dropout(p=0.2)
        self.out_linear = nn.Linear(self.bert.config.hidden_size, n_classes)
        self.softmax = nn.Softmax(dim=1)

    def forward(self, input_ids, attention_mask):
        _, pooled_out = self.bert(
            input_ids=input_ids,
            attention_mask=attention_mask
        )
        out = self.dropout(pooled_out)
        out = self.out_linear(out)
        return self.softmax(out)<jupyter_output><empty_output><jupyter_text>serving interfaceThe load function essentialy instantiates our custom model with the architecture defined above.<jupyter_code>import mlrun
class SentimentClassifierServing(mlrun.runtimes.MLModelServer):
    def load(self):
        model_file, _ = self.get_model('.pt')
        device = torch.device('cuda:0') if torch.cuda.is_available() else torch.device('cpu')
        model = BertSentimentClassifier(n_classes=3)
        model.load_state_dict(torch.load(model_file, map_location=device))
        model.eval()
        self.model = model
    def predict(self, body):
        try:
            instances = body['instances']
            enc = tokenizer.batch_encode_plus(instances, return_tensors='pt', pad_to_max_length=True)
            outputs = self.model(input_ids=enc['input_ids'], attention_mask=enc['attention_mask'])
            _, preds = torch.max(outputs, dim=1)
            return preds.cpu().tolist()
        except Exception as e:
            raise Exception("Failed to predict %s" % e)
# nuclio: end-code<jupyter_output><empty_output><jupyter_text>mlconfig<jupyter_code>from mlrun import mlconf
import os

mlconf.dbpath = mlconf.dbpath or 'http://mlrun-api:8080'
mlconf.artifact_path = mlconf.artifact_path or f'{os.environ["HOME"]}/artifacts'<jupyter_output><empty_output><jupyter_text>test locallyYou may change model_dir to point at the path where model.pt file is saved<jupyter_code>model_dir = '/User/demo-stocks/artifacts/models/bert_sentiment_analysis_model.pt'
model_server = SentimentClassifierServing('model-server', model_dir=model_dir)
model_server.load()<jupyter_output><empty_output><jupyter_text>test 1Here we test a pretty straightforward example for positive sentiment.<jupyter_code>output = model_server.predict({"instances":['I had a pleasure to work with such dedicated team. Looking forward to \
             cooperate with each and every one of them again.']})

assert output[0] == 2<jupyter_output><empty_output><jupyter_text>test 2Now we will test a couple more examples. These are arguably harder due to misleading words that express, on their own, an opposite sentiment comparing to the full text.<jupyter_code>output = model_server.predict({"instances":['This app is amazingly useless.',
                     'As much as I hate to admit it, the new added feature is surprisingly user friendly.']})

assert output[0] == 0
assert output[1] == 2<jupyter_output><empty_output><jupyter_text>remote activationCreate a function object with custom specification.<jupyter_code>from mlrun import new_model_server, mount_v3io
import requests
fn = new_model_server('sentiment-analysis-serving',
                      model_class='SentimentClassifierServing')
fn.spec.description = "BERT based sentiment classification model"
fn.metadata.categories = ['serving', 'NLP', 'BERT', 'sentiment analysis']
fn.metadata.labels = {'author': 'roye', 'framework': "pytorch"}
fn.spec.max_replicas = 1
fn.export("function.yaml")

fn.add_model('bert_classifier_v1', '/User/demo-stocks/artifacts/models/bert_sentiment_analysis_model.pt')
if "V3IO_HOME" in list(os.environ):
    from mlrun import mount_v3io
    fn.apply(mount_v3io())
else:
    # is you set up mlrun using the instructions at
    # https://github.com/mlrun/mlrun/blob/master/hack/local/README.md
    from mlrun.platforms import mount_pvc
    fn.apply(mount_pvc('nfsvol', 'nfsvol', '/home/joyan/data'))
addr = fn.deploy(project='nlp-servers')<jupyter_output>> 2020-10-18 14:10:16,517 [info] deploy started
[nuclio] 2020-10-18 14:10:18,649 (info) Build complete
[nuclio] 2020-10-18 14:10:46,014 (info) Function deploy complete
[nuclio] 2020-10-18 14:10:46,022 done updating nlp-servers-sentiment-analysis-serving, function address: 192.168.224.209:31790
> 2020-10-18 14:10:46,032 [warning] warning!, server (0.5.1) and client (0.5.2) ver dont match<jupyter_text>remote testWe will send a sentence to the model server via HTTP request. Note that the url below uses model server notation that directs our event to the predict function.<jupyter_code>import json

event_data = {'instances': ['I had a somewhat ok experience buying at that store.']}

resp = requests.put(addr + '/bert_classifier_v1/predict', json=json.dumps(event_data))
print(resp.text)<jupyter_output>[1]
{'fraction_non_alphanumeric': 0.09954891896095816, 'fraction_numerical': 0.022865142323845077, 'mean_word_length': 4.834845735027224, 'pattern_counts': {'":': 2, '<': 34, '<?xml version=': 0, '>': 36, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 1, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '462509', 'license': 'Apache-2.0', 'n_tokens_mistral': 2098, 'n_tokens_neox': 1985, 'n_words': 539, 'path': 'sentiment_analysis_serving/bert_sentiment_analysis_serving.ipynb', 'repo_name': 'EdmondIguazio/functions', 'size': '6429'}