<jupyter_start><jupyter_text>Tasks and Classes Sub-task A :-Aggression Identification Shared Task. The task will be to develop a classifier that could make a 3-way classification in between ‘Overtly Aggressive’, ‘Covertly Aggressive’ and ‘Non-aggressive’ text data. We are making available a dataset of 5,000 aggression-annotated data from social media each in Bangla (in both Roman and Bangla script), Hindi (in both Roman and Devanagari script) and English for training and validation. We will release additional data for testing your system. The train and test sets for the tasks are different from the ones made available during TRAC - 1. Sub-task B :-Misogynistic Aggression Identification Shared Task: This task will be to develop a binary classifier for classifying the text as ‘gendered’ or ‘non-gendered’. We will provide a dataset of 5,000 annotated data from social media each in Bangla (in both Roman and Bangla script), Hindi (in both Roman and Devanagari script) and English for training and validation. We will release additional data for testing your system.* **(HATE) Hate speech** :- Posts under this class contain Hate speech content.* **(OFFN) Offenive** :- Posts under this class contain offensive content.* **(PRFN) Profane** :- These posts contain profane words. HATE SPEECHDescribing negative attributes or deficiencies to groups of individuals because they are members of a group (e.g. all poor people are stupid). Hateful comment toward groups because of race, political opinion, sexual orientation, gender, social status, health condition or similar. OFFENSIVEPosts which are degrading, dehumanizing,insulting an individual,threatening with violent acts are categorized into OFFENSIVE category. PROFANITYUnacceptable language in the absence of insults and abuse. This typically concerns the usage of swearwords (Scheiße, Fuck etc.) and cursing (Zur Hölle! Verdammt! etc.) are categorized into this category.We expect most posts to be OTHER, some to be HATE and the other two categories to be less frequent. Dubious cases which are difficult to decide even for humans, should be left out. Sub-task C :-Sub-task c will check the type of offense. Only posts labeled as HOF in sub-task A are included in sub-task C. The two categories in sub-task C are the following: * **Targeted Insult (TIN)**: Posts containing an insult/threat to an individual, group, or others.* **Untargeted (UNT)**: Posts containing nontargeted profanity and swearing. Posts with general profanity are not targeted, but they contain non-acceptable language.The multilingual HASOC Corpus will be sampled from Facebook and Twitter and distributed in tab separated format. Participants are allowed to use external resources and other datasets for this task. Dataset will be prepared in 3 languages (German, English and code-mixed hindi).The size of Training data corpus is approximately 8000 posts for each language and Test data is approximately 1000 posts for the each language. Classification systems in all tasks will be evaluated using either macro-averaged F1-score or weighted F1-score.<jupyter_code>%matplotlib inline

import os
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.feature_extraction.text import TfidfTransformer
from sklearn.linear_model import LogisticRegressionCV
import sklearn.metrics as sk_metrics
from sklearn.pipeline import Pipeline
from google.colab import drive
drive.mount('/content/gdrive')
%env TRAC_PATH /content/gdrive/My Drive/TRAC/
%%bash 
echo "${TRAC_PATH}"
ls -ltrh "${TRAC_PATH}/data"
realpath "${TRAC_PATH}"
ls -ltrh "${TRAC_PATH}"/data/**/*
BASE_PATH = os.environ.get("TRAC_PATH")

DATA_PATHS_TRAIN = {
    "EN": f"{BASE_PATH}/data/raw/train/trac2_eng_train.csv",
    "IBE": f"{BASE_PATH}/data/raw/train/trac2_iben_train.csv",
    "HI": f"{BASE_PATH}/data/raw/train/trac2_hin_train.csv"
}
DATA_PATHS_DEV = {
    "EN": f"{BASE_PATH}/data/raw/dev/trac2_eng_dev.csv",
    "IBE": f"{BASE_PATH}/data/raw/dev/trac2_iben_dev.csv",
    "HI": f"{BASE_PATH}/data/raw/dev/trac2_hin_dev.csv"
}
DATA_PATHS_TEST = {
    "EN": f"{BASE_PATH}/data/raw/test/trac2_eng_test.csv",
    "IBE": f"{BASE_PATH}/data/raw/test/trac2_iben_test.csv",
    "HI": f"{BASE_PATH}/data/raw/test/trac2_hin_test.csv"
}

#print(DATA_PATHS_DEV)

DATA_COLUMNS = ["row_id", "text", "task_1", "task_2"]

NUM_LANGUAGES = len(DATA_PATHS_TRAIN)
print(NUM_LANGUAGES)
TASK_LABEL_IDS = {
    "Sub-task A": ["OAG", "NAG", "CAG"],
    "Sub-task B": ["GEN", "NGEN"]
}

df = pd.read_csv(DATA_PATHS_TRAIN["EN"], sep=",").fillna("NULL")
df.head()
for data_type, DATA_PATHS in [("train", DATA_PATHS_TRAIN), ("dev", DATA_PATHS_DEV)]:
    print(data_type)
    for lang, path in DATA_PATHS.items():
        df = pd.read_csv(path, sep=",").fillna("NULL")
        task_cols = df.filter(regex=r'Sub-task*', axis=1).columns
        print(df.columns)
        n_cols = len(task_cols)
        fig, ax = plt.subplots(1, n_cols, sharey=True, figsize=(5*n_cols, 4))
        for axi, task in zip(ax.flatten(), task_cols):
            y = df[task].str.upper().replace("NULL", "NONE")
            sns.countplot(y, order=sorted(y.unique()), ax=axi, color="0.3")
        fig.suptitle(f"data_type={data_type}, lang={lang}")
        sns.despine(offset=10)
scorer = sk_metrics.make_scorer(sk_metrics.f1_score, average="macro")
from sklearn import preprocessing
for lang in ["EN", "IBE", "HI"]:
    df_train=pd.read_csv(DATA_PATHS_TRAIN[lang], sep=',').fillna("NULL")
    df_dev=pd.read_csv(DATA_PATHS_DEV[lang], sep=',').fillna("NULL")
    X_train = df_train["Text"]
    X_dev = df_dev["Text"]
    print(f"{'='*20} {lang} {'='*20}")
    for task in ["Sub-task A", "Sub-task B"]:
        if task not in df_train.columns:
            continue
        print(f"{'-'*20} {task} {'-'*20}")
        y_train = df_train[task].str.upper().replace("NULL", "NONE")
        y_dev = df_dev[task].str.upper().replace("NULL", "NONE")
        clf = Pipeline([ 
            ('vect', CountVectorizer()),
            ('tfidf', TfidfTransformer()),
            ('clf', LogisticRegressionCV(n_jobs=2, multi_class="multinomial", scoring=scorer,max_iter=300)),
        ])
        idx = y_train != "NONE"
        clf.fit(X_train[idx], y_train[idx])
        y_pred = clf.predict(X_train)
        print(f"accuracy[train]\t= {sk_metrics.accuracy_score(y_train[idx], y_pred[idx]):.3f}")
        print(sk_metrics.classification_report(y_train[idx], y_pred[idx]))
        idx = y_dev != "NONE"
        y_pred = clf.predict(X_dev)
        print(f"accuracy[dev]\t= {sk_metrics.accuracy_score(y_dev[idx], y_pred[idx]):.3f}")
        print(sk_metrics.classification_report(y_dev[idx], y_pred[idx]))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11889250814332247, 'fraction_numerical': 0.008143322475570033, 'mean_word_length': 4.482954545454546, 'pattern_counts': {'":': 11, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '95919', 'license': 'MIT', 'n_tokens_mistral': 2230, 'n_tokens_neox': 2139, 'n_words': 677, 'path': 'notebooks/Training_data_sklearn_models.ipynb', 'repo_name': 'socialmediaie/TRAC2020', 'size': '6754'}