---
license: other
base_model: microsoft/phi-1_5
tags:
- generated_from_trainer
model-index:
- name: titletor-phi_1-5
  results: []
---

<div align="center">

# Titletor

</div>


<div align="center">
  <img src="./titletor.png" width="300"/>
</div>

This model is a fine-tuned version of [microsoft/phi-1_5](https://huggingface.co/microsoft/phi-1_5) on [zelalt/scientific-papers-3.5-withprompt](https://huggingface.co/datasets/zelalt/scientific-papers-3.5-withprompt) dataset.
It achieves the following results on the evaluation set:
- Loss: 2.1587

## Model description

### Sample Code

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("zelalt/titletor-phi_1-5", trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained("zelalt/titletor-phi_1-5", trust_remote_code=True)
inputs = tokenizer(f'''What is the title of this paper? ....[your pdf as text]\n\nAnswer: ''', return_tensors="pt", return_attention_mask=False)
outputs = model.generate(**inputs,max_new_tokens=50, pad_token_id = tokenizer.eos_token_id, eos_token_id = tokenizer.eos_token_id)
text = tokenizer.batch_decode(outputs)[0]
print(text)
```

## Training and evaluation data
Train and validation dataset:
[zelalt/scientific-papers-3.5-withprompt](https://huggingface.co/datasets/zelalt/scientific-papers-3.5-withprompt)


## Training procedure

### Training hyperparameters

- total_train_batch_size: 8
- lr_scheduler_type: cosine

### Training results

| Training Loss | Epoch | Step | Validation Loss |
|:-------------:|:-----:|:----:|:---------------:|
| 1.8515        | 2.94  | 1160 | 2.1587          |


### Framework versions

- Transformers 4.35.2
- Pytorch 2.1.0+cu118
- Datasets 2.15.0
- Tokenizers 0.15.0