Edit model card

bart-large-finetuned-arxiv-co-ga-latest

Model description

This model (v1.0) is a fine-tuned version of facebook/bart-large. The purpose of this model is to generate titles given an abstract. It was trained on Astronomy arXiv papers tagged 'CO' (Cosmology and Nongalactic Astrophysics) as well as 'GA' (Astrophysics of Galaxies).

Code for this project can be found on GitHub.

👉🏽 Feel free to interact with the model here and use it to generate a title given your abstract! 👈🏽

Training and evaluation data

The dataset used for training consists of abstract+title pairs from arXiv and was obtained from Kaggle. Training was performed on 79,727 abstract+title pairs and validation was done on 9966 abstract+title pairs.

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 1
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Rouge1 Rouge2 Rougel Rougelsum
1.7752 1.0 9966 1.7190 43.8916 23.6296 38.229 39.3519

Framework versions

  • Transformers 4.28.0
  • Pytorch 2.0.1+cu118
  • Datasets 2.12.0
  • Tokenizers 0.13.3
Downloads last month
27
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Dataset used to train mehnaazasad/bart-large-finetuned-arxiv-co-ga-latest

Space using mehnaazasad/bart-large-finetuned-arxiv-co-ga-latest 1