<jupyter_start><jupyter_text>Subjectivity Analysis This tutorial is available as an IPython notebook at [Malaya/example/subjectivity](https://github.com/huseinzol05/Malaya/tree/master/example/subjectivity).     This module trained on both standard and local (included social media) language structures, so it is save to use for both.<jupyter_code>%%time
import malaya<jupyter_output>CPU times: user 5.2 s, sys: 869 ms, total: 6.07 s
Wall time: 6.23 s<jupyter_text>ExplanationPositive subjectivity: based on or influenced by personal feelings, tastes, or opinions. Can be a positive or negative sentiment.Negative subjectivity: based on a report or a fact. Can be a positive or negative sentiment.<jupyter_code>negative_text = 'Kerajaan negeri Kelantan mempersoalkan motif kenyataan Menteri Kewangan Lim Guan Eng yang hanya menyebut Kelantan penerima terbesar bantuan kewangan dari Kerajaan Persekutuan. Sedangkan menurut Timbalan Menteri Besarnya, Datuk Mohd Amar Nik Abdullah, negeri lain yang lebih maju dari Kelantan turut mendapat pembiayaan dan pinjaman.'
positive_text = 'kerajaan sebenarnya sangat bencikan rakyatnya, minyak naik dan segalanya'

string1 = 'Sis, students from overseas were brought back because they are not in their countries which is if something happens to them, its not the other countries’ responsibility. Student dalam malaysia ni dah dlm tggjawab kerajaan. Mana part yg tak faham?'
string2 = 'Harap kerajaan tak bukak serentak. Slowly release week by week. Focus on economy related industries dulu'
string3 = 'Idk if aku salah baca ke apa. Bayaran rm350 utk golongan umur 21 ke bawah shj ? Anyone? If 21 ke atas ok lah. If umur 21 ke bawah?  Are you serious? Siapa yg lebih byk komitmen? Aku hrp aku salah baca. Aku tk jumpa artikel tu'
string4 = 'Jabatan Penjara Malaysia diperuntukkan RM20 juta laksana program pembangunan Insan kepada banduan. Majikan yang menggaji bekas banduan, bekas penagih dadah diberi potongan cukai tambahan sehingga 2025.'<jupyter_output><empty_output><jupyter_text>Load multinomial modelAll model interface will follow sklearn interface started v3.4,```pythonmodel.predict(List[str])model.predict_proba(List[str])```<jupyter_code>model = malaya.subjectivity.multinomial()
model.predict_proba([positive_text,negative_text])
model.predict_proba([positive_text,negative_text], add_neutral = False)<jupyter_output><empty_output><jupyter_text>List available Transformer models<jupyter_code>malaya.subjectivity.available_transformer()<jupyter_output>INFO:root:tested on 20% test set.<jupyter_text>Make sure you can check accuracy chart from here first before select a model, https://malaya.readthedocs.io/en/latest/Accuracy.htmlsubjectivity-analysis**You might want to use Tiny-Albert, a very small size, 22.4MB, but the accuracy is still on the top notch.**  Load ALBERT modelAll model interface will follow sklearn interface started v3.4,```pythonmodel.predict(List[str])model.predict_proba(List[str])```<jupyter_code>model = malaya.subjectivity.transformer(model = 'albert')<jupyter_output>INFO:tensorflow:loading sentence piece model<jupyter_text>Load Quantized modelTo load 8-bit quantized model, simply pass `quantized = True`, default is `False`.We can expect slightly accuracy drop from quantized model, and not necessary faster than normal 32-bit float model, totally depends on machine.<jupyter_code>quantized_model = malaya.subjectivity.transformer(model = 'albert', quantized = True)<jupyter_output>WARNING:root:Load quantized model will cause accuracy drop.<jupyter_text>Predict batch of strings<jupyter_code>model.predict_proba([negative_text, positive_text])
quantized_model.predict_proba([negative_text, positive_text])<jupyter_output><empty_output><jupyter_text>Open subjectivity visualization dashboardDefault when you call `predict_words` it will open a browser with visualization dashboard, you can disable by `visualization=False`.<jupyter_code>model.predict_words(negative_text)
from IPython.core.display import Image, display

display(Image('subjective-dashboard.png', width=800))<jupyter_output><empty_output><jupyter_text>VectorizeLet say you want to visualize sentence / word level in lower dimension, you can use `model.vectorize`,```pythondef vectorize(self, strings: List[str], method: str = 'first'):    """    vectorize list of strings.    Parameters    ----------    strings: List[str]    method : str, optional (default='first')        Vectorization layer supported. Allowed values:        * ``'last'`` - vector from last sequence.        * ``'first'`` - vector from first sequence.        * ``'mean'`` - average vectors from all sequences.        * ``'word'`` - average vectors based on tokens.    Returns    -------    result: np.array    """```  Sentence level<jupyter_code>texts = [negative_text, positive_text, string1, string2]
r = quantized_model.vectorize(texts, method = 'first')
from sklearn.manifold import TSNE
import matplotlib.pyplot as plt

tsne = TSNE().fit_transform(r)
tsne.shape
plt.figure(figsize = (7, 7))
plt.scatter(tsne[:, 0], tsne[:, 1])
labels = texts
for label, x, y in zip(
    labels, tsne[:, 0], tsne[:, 1]
):
    label = (
        '%s, %.3f' % (label[0], label[1])
        if isinstance(label, list)
        else label
    )
    plt.annotate(
        label,
        xy = (x, y),
        xytext = (0, 0),
        textcoords = 'offset points',
    )<jupyter_output><empty_output><jupyter_text>Word level<jupyter_code>r = quantized_model.vectorize(texts, method = 'word')
x, y = [], []
for row in r:
    x.extend([i[0] for i in row])
    y.extend([i[1] for i in row])
tsne = TSNE().fit_transform(y)
tsne.shape
plt.figure(figsize = (7, 7))
plt.scatter(tsne[:, 0], tsne[:, 1])
labels = x
for label, x, y in zip(
    labels, tsne[:, 0], tsne[:, 1]
):
    label = (
        '%s, %.3f' % (label[0], label[1])
        if isinstance(label, list)
        else label
    )
    plt.annotate(
        label,
        xy = (x, y),
        xytext = (0, 0),
        textcoords = 'offset points',
    )<jupyter_output><empty_output><jupyter_text>Pretty good, the model able to know cluster top side as positive subjectivity, bottom side as negative subjectivity.  Stacking modelsMore information, you can read at [https://malaya.readthedocs.io/en/latest/Stack.html](https://malaya.readthedocs.io/en/latest/Stack.html)<jupyter_code>multinomial = malaya.subjectivity.multinomial()
alxlnet = malaya.subjectivity.transformer(model = 'alxlnet')
malaya.stack.predict_stack([multinomial, model, alxlnet], [positive_text])
malaya.stack.predict_stack([multinomial, model, alxlnet], [positive_text], add_neutral = False)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10334438083760168, 'fraction_numerical': 0.010997288339861404, 'mean_word_length': 5.085242896425298, 'pattern_counts': {'":': 0, '<': 41, '<?xml version=': 0, '>': 41, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '263362', 'license': 'MIT', 'n_tokens_mistral': 2198, 'n_tokens_neox': 2100, 'n_words': 645, 'path': 'docs/load-subjectivity.ipynb', 'repo_name': 'illaiza115/malaya', 'size': '6638'}