task_categories:text-classification task_categories:text2text-generation task_categories:fill-mask size_categories:100K<n<1M language:en license:mit masked mask-scored comment scoring masked-model region:us
---
dataset_info:
  features:
  - name: text
    dtype: string
  - name: text_masked
    dtype: string
  - name: text_replaced
    list:
    - name: score
      dtype: float64
    - name: sequence
      dtype: string
    - name: token
      dtype: int64
    - name: token_str
      dtype: string
  splits:
  - name: train
    num_bytes: 872262083
    num_examples: 451219
  download_size: 333147199
  dataset_size: 872262083
configs:
- config_name: default
  data_files:
  - split: train
    path: data/train-*
license: mit
task_categories:
- text-classification
- text2text-generation
- fill-mask
language:
- en
tags:
- masked
- mask-scored
- comment scoring
- masked-model
pretty_name: civil comments w/hatebert scoring
size_categories:
- 100K<n<1M
---
# Dataset Card for "civil_comments_hatebert"

This is an experiment to see how "civil-comments" can be changed by models without much manipulation to offensive speech in certain cases.

This data is a reformat of the civil comments dataset, discarding all scoring attributes of abusive speech, masking random tokens, and processing with hatebert to fill-masked tokens with possible abusive language. 
This merely sets up some good data for three things: fill-mask activities, text training, and scored responses based on random tokens being manipulatible according to this model. 
Showing the progress of incarnation, three columns illustrate the original text data extracted, the randomly masked text, and the filled text with scores in a list for the hatebert output. 
So far in practice, the hatebert model mostly fills with innocuous placeholders, from *very* limited testing.

Hatebert is as it sounds, a BERT based model trained on fill-mask activites. 


[civil_comments dataset](https://huggingface.co/datasets/civil_comments)
[hatebert model](https://huggingface.co/datasets/civil_comments)


[More Information needed](https://github.com/huggingface/datasets/blob/main/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)