task_categories:summarization size_categories:10K<n<100K language:en license:cc0-1.0 region:us
---
license: cc0-1.0
task_categories:
- summarization
language:
- en
pretty_name: ML Articles Subset of Scientific Papers
size_categories:
- 10K<n<100K
---

# Dataset Card for 'ML Articles Subset of Scientific Papers' Dataset
## Dataset Summary
The dataset consists of 32,621 instances from the 'Scientific papers' dataset, a selection of scientific papers and summaries from ArXiv repository. This subset focuses on articles that are semantically, vocabulary-wise, structurally, and meaningfully closest to articles describing machine learning. This subset was created using sentence embeddings and K-means clustering.

## Supported Tasks and Leaderboards
The dataset supports tasks related to text summarization. Particularly, the dataset was created for fine-tuning transformer models for summarization. There are no established leaderboards at this moment.

## Languages
The text in the dataset is in English.

## Dataset Structure

### Data Instances
An instance in the dataset includes a scientific paper and its summary, both in English.

### Data Fields

article: The full text of the scientific paper.\
abstract: The summary of the paper.

### Data Splits
The dataset is split into:\
-training subset: 30280 articles\
-validation subset: 1196 articles\
-test subset: 1145 articles

## Dataset Creation

### Methods
The subset was created using sentence embeddings from a transformer model, SciBERT. The embeddings were clustered into 6 clusters using the K-means clustering algorithm. The cluster closest to articles strongly related to the machine learning area by cosine similarity was chosen to form this dataset.

### Source Data
The dataset is a subset of the 'Scientific papers' dataset, which includes scientific papers from the ArXiv repository.

### Social Impact
This dataset could help improve the quality of summarization models for machine learning research articles, which in turn can make such content more accessible.

### Discussion of Biases
As the dataset focuses on machine learning articles, it may not be representative of scientific papers in general or other specific domains.

### Other Known Limitations
As the dataset has been selected based on a specific methodology, it may not include all machine learning articles or may inadvertently include non-machine learning articles.

### Dataset Curators
The subset was created as part of a project aimed to build an effective summarization model for Machine Learning articles.