<jupyter_start><jupyter_text>MIT LicenseCopyright (c) 2021  <NAME>,                    <NAME>,                    <NAME>,                    <NAME>,                    <NAME>,                    <NAME>,                    <NAME>Permission is hereby granted, free of charge, to any person obtaining a copyof this software and associated documentation files (the "Software"), to dealin the Software without restriction, including without limitation the rightsto use, copy, modify, merge, publish, distribute, sublicense, and/or sellcopies of the Software, and to permit persons to whom the Software isfurnished to do so, subject to the following conditions:The above copyright notice and this permission notice shall be included in allcopies or substantial portions of the Software.THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS ORIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THEAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHERLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THESOFTWARE.  Sesgos de tweets cambiando partidos politicosEn este notebook se analiza los csv *tweet_mods_political_parties_[PARTY].csv* según la toxicidad "toxic" y "very_toxic" del modelo *finetuned-toxic-political-tweets-es* de Newtral<jupyter_code>import pandas as pd
import numpy as np
if 'google.colab' in str(get_ipython()):
  from google.colab import drive
  drive.mount('/content/drive')
  PP_tweets = pd.read_csv('/content/drive/Shareddrives/ETICA/TweetMod_csv/tweet_mods_political_parties_PP.csv')
  PSOE_tweets = pd.read_csv('/content/drive/Shareddrives/ETICA/TweetMod_csv/tweet_mods_political_parties_PSOE.csv')
  VOX_tweets = pd.read_csv('/content/drive/Shareddrives/ETICA/TweetMod_csv/tweet_mods_political_parties_VOX.csv')
  Vox_tweets = pd.read_csv('/content/drive/Shareddrives/ETICA/TweetMod_csv/tweet_mods_political_parties_Vox.csv')
  PODEMOS_tweets = pd.read_csv('/content/drive/Shareddrives/ETICA/TweetMod_csv/tweet_mods_political_parties_PODEMOS.csv')
  Unidas_Podemos_tweets = pd.read_csv('/content/drive/Shareddrives/ETICA/TweetMod_csv/tweet_mods_political_parties_Unidas_Podemos.csv')
  Ciudadanos_tweets = pd.read_csv('/content/drive/Shareddrives/ETICA/TweetMod_csv/tweet_mods_political_parties_Ciudadanos.csv')
  Cs_tweets = pd.read_csv('/content/drive/Shareddrives/ETICA/TweetMod_csv/tweet_mods_political_parties_Cs.csv')
else:
  PP_tweets = pd.read_csv('../data/tweet_mods_political_parties_PP.csv')
  PSOE_tweets = pd.read_csv('../data/tweet_mods_political_parties_PSOE.csv')
  VOX_tweets = pd.read_csv('../data/tweet_mods_political_parties_VOX.csv')
  Vox_tweets = pd.read_csv('../data/tweet_mods_political_parties_Vox.csv')
  PODEMOS_tweets = pd.read_csv('../data/tweet_mods_political_parties_PODEMOS.csv')
  Unidas_Podemos_tweets = pd.read_csv('../data/tweet_mods_political_parties_Unidas_Podemos.csv')
  Ciudadanos_tweets = pd.read_csv('../data/tweet_mods_political_parties_Ciudadanos.csv')
  Cs_tweets = pd.read_csv('../data/tweet_mods_political_parties_Cs.csv')
palabras_politicas = ["PP", "PSOE", "VOX", "PODEMOS", "Vox", "Cs", "Ciudadanos", "Unidas Podemos", "Partido popular"]
tox_dict = { i : 0 for i in palabras_politicas }

def variacionToxicidad(df, var=0.10, very_tox=0, key_words=palabras_politicas, tox_dict=tox_dict):
  var_df = df.copy()
  var_df.drop(var_df.index, inplace=True)

  for idx, row in df.iterrows():
    if very_tox == 0:
      if row.most_toxic_party_toxicity > (row.toxic_original + var):
        var_df = var_df.append(df.iloc[idx])
    else:
      if row.most_verytoxic_party_toxicity > (row.very_toxic_original + var):
        var_df = var_df.append(df.iloc[idx])

  print('\t\t Tweets totales: ', var_df.shape[0])
  print('\t\t Cambiando la mención por:')
  for p in key_words:
    print('\t\t\t -', p, ': ', var_df[var_df['most_toxic_party'] == p].shape[0])
    tox_dict.update({p: tox_dict.get(p) + var_df[var_df['most_toxic_party'] == p].shape[0]})

  return var_df<jupyter_output><empty_output><jupyter_text>Vamos a ver como ha cambiado la toxicidad de un tweet sustituyendo la mención a un partido por una mención a otro partido distinto:<jupyter_code>print('Menciones PP')
print('\t Aumento de Toxic:')
PP_variacion_tox = variacionToxicidad(PP_tweets)
print('\t Aumento de Very Toxic:')
PP_variacion_verytox = variacionToxicidad(PP_tweets, very_tox=1)

print('Menciones PSOE')
print('\t Aumento de Toxic:')
PSOE_variacion_tox = variacionToxicidad(PSOE_tweets)
print('\t Aumento de Very Toxic:')
PSOE_variacion_verytox = variacionToxicidad(PSOE_tweets, very_tox=1)

print('Menciones VOX')
print('\t Aumento de Toxic:')
VOX_variacion_tox = variacionToxicidad(VOX_tweets)
print('\t Aumento de Very Toxic:')
VOX_variacion_verytox = variacionToxicidad(VOX_tweets, very_tox=1)

print('Menciones Vox')
print('\t Aumento de Toxic:')
Vox_variacion_tox = variacionToxicidad(Vox_tweets)
print('\t Aumento de Very Toxic:')
Vox_variacion_verytox = variacionToxicidad(Vox_tweets, very_tox=1)

print('Menciones PODEMOS')
print('\t Aumento de Toxic:')
PODEMOS_variacion_tox = variacionToxicidad(PODEMOS_tweets)
print('\t Aumento de Very Toxic:')
PODEMOS_variacion_verytox = variacionToxicidad(PODEMOS_tweets, very_tox=1)

print('Menciones Unidas Podemos')
print('\t Aumento de Toxic:')
Unidas_Podemos_variacion_tox = variacionToxicidad(Unidas_Podemos_tweets)
print('\t Aumento de Very Toxic:')
Unidas_Podemos_variacion_verytox = variacionToxicidad(Unidas_Podemos_tweets, very_tox=1)

print('Menciones Ciudadanos')
print('\t Aumento de Toxic:')
Ciudadanos_variacion_tox = variacionToxicidad(Ciudadanos_tweets)
print('\t Aumento de Very Toxic:')
Ciudadanos_variacion_verytox = variacionToxicidad(Ciudadanos_tweets, very_tox=1)

print('Menciones Cs')
print('\t Aumento de Toxic:')
Cs_variacion_tox = variacionToxicidad(Cs_tweets)
print('\t Aumento de Very Toxic:')
Cs_variacion_verytox = variacionToxicidad(Cs_tweets, very_tox=1)
datasets_tox = [PP_variacion_tox, PSOE_variacion_tox, VOX_variacion_tox, Vox_variacion_tox, PODEMOS_variacion_tox, Unidas_Podemos_variacion_tox, Ciudadanos_variacion_tox, Cs_variacion_tox]
datasets_verytox = [PP_variacion_verytox, PSOE_variacion_verytox, VOX_variacion_verytox, Vox_variacion_verytox, PODEMOS_variacion_verytox, Unidas_Podemos_variacion_verytox, Ciudadanos_variacion_verytox, Cs_variacion_verytox]
!python --version<jupyter_output>Python 3.9.7<jupyter_text>Vemos cuáles son las menciones que han aumentado más la toxicidad de los tweets:<jupyter_code>sorted(tox_dict.items(), key=lambda x: x[1], reverse=True)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1021584861028977, 'fraction_numerical': 0.0036960378474275575, 'mean_word_length': 6.189160467587673, 'pattern_counts': {'":': 0, '<': 19, '<?xml version=': 0, '>': 21, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '269053', 'license': 'MIT', 'n_tokens_mistral': 2667, 'n_tokens_neox': 2487, 'n_words': 493, 'path': 'analysis_notebooks/Sesgos-partidos.ipynb', 'repo_name': 'AlbertoGarciaDomenech/SesgosTocicidadTweets', 'size': '6880'}