
# bart-large-finetuned-arxiv-co-ga-latest

## Model description

This model (v1.0) is a fine-tuned version of [facebook/bart-large](https://huggingface.co/facebook/bart-large). The purpose of this model is 
to generate titles given an abstract. It was trained on Astronomy arXiv papers tagged 'CO' (Cosmology and Nongalactic Astrophysics) 
as well as 'GA' (Astrophysics of Galaxies).

Code for this project can be found on [GitHub](https://github.com/MehnaazAsad/NLP_summarization_bart). 

👉🏽 Feel free to interact with the model [here](https://huggingface.co/spaces/mehnaazasad/give-me-a-title) and use it to generate a title given your abstract! 👈🏽

<!-- ## Intended uses & limitations

More information needed -->

## Training and evaluation data

The dataset used for training consists of abstract+title pairs from arXiv and was obtained from 
[Kaggle](https://www.kaggle.com/datasets/Cornell-University/arxiv/code). Training was performed on 79,727 abstract+title pairs and 
validation was done on 9966 abstract+title pairs.

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 1
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss | Rouge1  | Rouge2  | Rougel | Rougelsum |
|:-------------:|:-----:|:----:|:---------------:|:-------:|:-------:|:------:|:---------:|
| 1.7752        | 1.0   | 9966 | 1.7190          | 43.8916 | 23.6296 | 38.229 | 39.3519   |


### Framework versions

- Transformers 4.28.0
- Pytorch 2.0.1+cu118
- Datasets 2.12.0
- Tokenizers 0.13.3
{"license": "apache-2.0", "tags": ["generated_from_trainer"], "datasets": ["mehnaazasad/arxiv_astro_co_ga"], "metrics": ["rouge"], "model-index": [{"name": "bart-large-finetuned-arxiv-co-ga-latest", "results": []}]}
transformers pytorch tensorboard bart text2text-generation generated_from_trainer dataset:mehnaazasad/arxiv_astro_co_ga license:apache-2.0 autotrain_compatible endpoints_compatible has_space region:us
TAGS
#transformers #pytorch #tensorboard #bart #text2text-generation #generated_from_trainer #dataset-mehnaazasad/arxiv_astro_co_ga #license-apache-2.0 #autotrain_compatible #endpoints_compatible #has_space #region-us 

bart-large-finetuned-arxiv-co-ga-latest
=======================================


Model description
-----------------


This model (v1.0) is a fine-tuned version of facebook/bart-large. The purpose of this model is
to generate titles given an abstract. It was trained on Astronomy arXiv papers tagged 'CO' (Cosmology and Nongalactic Astrophysics)
as well as 'GA' (Astrophysics of Galaxies).


Code for this project can be found on GitHub.


Feel free to interact with the model here and use it to generate a title given your abstract!


Training and evaluation data
----------------------------


The dataset used for training consists of abstract+title pairs from arXiv and was obtained from
Kaggle. Training was performed on 79,727 abstract+title pairs and
validation was done on 9966 abstract+title pairs.


Training procedure
------------------


### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 2e-05
* train\_batch\_size: 8
* eval\_batch\_size: 8
* seed: 42
* optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
* lr\_scheduler\_type: linear
* num\_epochs: 1
* mixed\_precision\_training: Native AMP


### Training results



### Framework versions


* Transformers 4.28.0
* Pytorch 2.0.1+cu118
* Datasets 2.12.0
* Tokenizers 0.13.3



### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 2e-05
* train\_batch\_size: 8
* eval\_batch\_size: 8
* seed: 42
* optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
* lr\_scheduler\_type: linear
* num\_epochs: 1
* mixed\_precision\_training: Native AMP ### Training results ### Framework versions


* Transformers 4.28.0
* Pytorch 2.0.1+cu118
* Datasets 2.12.0
* Tokenizers 0.13.3
TAGS
#transformers #pytorch #tensorboard #bart #text2text-generation #generated_from_trainer #dataset-mehnaazasad/arxiv_astro_co_ga #license-apache-2.0 #autotrain_compatible #endpoints_compatible #has_space #region-us 
 ### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 2e-05
* train\_batch\_size: 8
* eval\_batch\_size: 8
* seed: 42
* optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
* lr\_scheduler\_type: linear
* num\_epochs: 1
* mixed\_precision\_training: Native AMP ### Training results ### Framework versions


* Transformers 4.28.0
* Pytorch 2.0.1+cu118
* Datasets 2.12.0
* Tokenizers 0.13.3