---
license: apache-2.0
base_model: tiiuae/falcon-7b
tags:
- generated_from_trainer
model-index:
- name: falcon-rewrite-560
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# falcon-rewrite-560

This model is a fine-tuned version of [tiiuae/falcon-7b](https://huggingface.co/tiiuae/falcon-7b) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 1.2253

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0005
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_ratio: 0.03
- num_epochs: 4

### Training results

| Training Loss | Epoch | Step | Validation Loss |
|:-------------:|:-----:|:----:|:---------------:|
| 3.4828        | 0.15  | 10   | 2.4554          |
| 3.8736        | 0.3   | 20   | 2.4426          |
| 3.6144        | 0.45  | 30   | 2.9606          |
| 1.9016        | 0.6   | 40   | 2.5989          |
| 1.8213        | 0.75  | 50   | 2.7219          |
| 3.2497        | 0.9   | 60   | 2.5531          |
| 4.2956        | 1.04  | 70   | 2.5117          |
| 2.0346        | 1.19  | 80   | 2.6559          |
| 3.293         | 1.34  | 90   | 2.5667          |
| 2.4628        | 1.49  | 100  | 2.7377          |
| 1.8473        | 1.64  | 110  | 1.9679          |
| 3.0982        | 1.79  | 120  | 2.4545          |
| 4.0502        | 1.94  | 130  | 1.5169          |
| 2.5271        | 2.09  | 140  | 2.2475          |
| 2.0131        | 2.24  | 150  | 2.2574          |
| 1.7571        | 2.39  | 160  | 1.9870          |
| 1.5135        | 2.54  | 170  | 2.4713          |
| 1.783         | 2.69  | 180  | 1.9229          |
| 1.4584        | 2.84  | 190  | 2.3683          |
| 0.7306        | 2.99  | 200  | 1.2253          |
| 2.2726        | 3.13  | 210  | 1.8495          |
| 1.7065        | 3.28  | 220  | 1.7916          |
| 0.5787        | 3.43  | 230  | 1.8749          |
| 1.3256        | 3.58  | 240  | 2.1531          |
| 2.8024        | 3.73  | 250  | 2.1011          |
| 1.1855        | 3.88  | 260  | 2.1262          |


### Framework versions

- Transformers 4.32.1
- Pytorch 2.0.1
- Datasets 2.14.5
- Tokenizers 0.13.3