task_categories:conversational task_categories:text-generation task_categories:text2text-generation task_categories:translation task_categories:summarization language:ar region:us
---
language:
- ar
task_categories:
- conversational
- text-generation
- text2text-generation
- translation
- summarization
pretty_name: MAD
configs:
- config_name: default
  data_files:
  - split: train
    path: data/train-*
dataset_info:
  features:
  - name: GenId
    dtype: int64
  - name: SubId
    dtype: int64
  - name: DatasetName
    dtype: string
  - name: DatasetLink
    dtype: string
  - name: Text
    dtype: string
  - name: MetaData
    struct:
    - name: AboutAuthor
      dtype: string
    - name: AboutBook
      dtype: string
    - name: Author
      dtype: string
    - name: AuthorName
      dtype: string
    - name: BookLink
      dtype: string
    - name: BookName
      dtype: string
    - name: ChapterLink
      dtype: string
    - name: ChapterName
      dtype: string
    - name: Tags
      dtype: float64
    - name: __index_level_0__
      dtype: float64
    - name: created_date
      dtype: string
    - name: deleted
      dtype: bool
    - name: detoxify
      dtype: 'null'
    - name: emojis
      struct:
      - name: count
        sequence: int32
      - name: name
        sequence: string
    - name: id
      dtype: string
    - name: labels
      struct:
      - name: count
        sequence: int32
      - name: name
        sequence: string
      - name: value
        sequence: float64
    - name: lang
      dtype: string
    - name: message_id
      dtype: string
    - name: message_tree_id
      dtype: string
    - name: model_name
      dtype: 'null'
    - name: parent_id
      dtype: string
    - name: query_id
      dtype: string
    - name: rank
      dtype: float64
    - name: review_count
      dtype: float64
    - name: review_result
      dtype: bool
    - name: role
      dtype: string
    - name: synthetic
      dtype: bool
    - name: title
      dtype: string
    - name: tree_state
      dtype: string
    - name: url
      dtype: string
    - name: user_id
      dtype: string
  - name: ConcatenatedText
    dtype: int64
  - name: __index_level_0__
    dtype: float64
  splits:
  - name: train
    num_bytes: 1990497610
    num_examples: 131393
  download_size: 790648134
  dataset_size: 1990497610
---
# Dataset Card for "Mixed-Arabic-Dataset"

## Mixed Arabic Datasets (MAD)

The Mixed Arabic Datasets (MAD) project provides a comprehensive collection of diverse Arabic-language datasets, sourced from various repositories, platforms, and domains. These datasets cover a wide range of text types, including books, articles, Wikipedia content, stories, and more.

### MAD Repo vs. MAD Main

#### MAD Repo
- **Versatility**: In the MAD Repository (MAD Repo), datasets are made available in their original, native form. Researchers and practitioners can selectively download specific datasets that align with their specific interests or requirements.
- **Independent Access**: Each dataset is self-contained, enabling users to work with individual datasets independently, allowing for focused analyses and experiments.

#### MAD Main or simply MAD
- **Unified Dataframe**: MAD Main represents a harmonized and unified dataframe, incorporating all datasets from the MAD Repository. It provides a seamless and consolidated view of the entire MAD collection, making it convenient for comprehensive analyses and applications.
- **Holistic Perspective**: Researchers can access a broad spectrum of Arabic-language content within a single dataframe, promoting holistic exploration and insights across diverse text sources.

### Why MAD Main?
- **Efficiency**: Working with MAD Main streamlines the data acquisition process by consolidating multiple datasets into one structured dataframe. This is particularly beneficial for large-scale projects or studies requiring diverse data sources.
- **Interoperability**: With MAD Main, the datasets are integrated into a standardized format, enhancing interoperability and compatibility with a wide range of data processing and analysis tools.
- **Meta-Analysis**: Researchers can conduct comprehensive analyses, such as cross-domain studies, trend analyses, or comparative studies, by leveraging the combined richness of all MAD datasets.

### Getting Started
- To access individual datasets in their original form, refer to the MAD Repository ([Link to MAD Repo](https://huggingface.co/datasets/M-A-D/Mixed-Arabic-Datasets-Repo)).
- For a unified view of all datasets, conveniently organized in a dataframe, you are here in the right place.
```python
from datasets import load_dataset

dataset = load_dataset("M-A-D/Mixed-Arabic-Dataset-Main")
```

### Join Us on Discord

For discussions, contributions, and community interactions, join us on Discord! [![Discord](https://img.shields.io/discord/798499298231726101?label=Join%20us%20on%20Discord&logo=discord&logoColor=white&style=for-the-badge)](https://discord.gg/2NpJ9JGm)

### How to Contribute

Want to contribute to the Mixed Arabic Datasets project? Follow our comprehensive guide on Google Colab for step-by-step instructions: [Contribution Guide](https://colab.research.google.com/drive/1w7_7lL6w7nM9DcDmTZe1Vfiwkio6SA-w?usp=sharing).

**Note**: If you'd like to test a contribution before submitting it, feel free to do so on the [MAD Test Dataset](https://huggingface.co/datasets/M-A-D/Mixed-Arabic-Dataset-test).

## Citation

```
@dataset{ 
title = {Mixed Arabic Datasets (MAD)},
author = {MAD Community},
howpublished = {Dataset},
url = {https://huggingface.co/datasets/M-A-D/Mixed-Arabic-Datasets-Repo},
year = {2023},
}
```