---
base_model:
- google-t5/t5-small
language:
- en
library_name: transformers
tags:
- Summarization
---

## Model Description

t5-small-amazon-reviews-summarization-finetuned-8bit-lora is a fine-tuned version of the T5-small model, adapted for the task of summarizing Amazon product reviews. The model has been fine-tuned using Low-Rank Adaptation (LoRA) techniques and 8-bit quantization, allowing for efficient training and inference while maintaining performance.

### Model Architecture
  <ul>
    <li>Base Model: T5-small</li>
    <li>Architecture: Encoder-Decoder Transformer</li>
    <li>Parameters: Approximately 60 million parameters</li>
    <li>Fine-tuning Method: LoRA (Low-Rank Adaptation) via the PEFT library</li>
    <li>Quantization: 8-bit quantization using the bitsandbytes library</li>
  </ul>
  
## Training Data

The model was fine-tuned on a dataset of Amazon product reviews on products of the beauty category, with corresponding summaries.
  <ul>
    <li>Dataset Source: Custom dataset derived from Amazon product reviews</li>
    <li>Dataset Size: 19665 pairs</li>
    <li>Dataset Content: Pairs of product reviews and their corresponding summaries</li>
    <li>Dataset Split: 80% training, 20% validation</li>
  </ul>
  
## Training Procedure

### Preprocessing
  <ul>
    Input Preparation:
        <li>Each input review was prefixed with "summarize: " to adhere to T5's format for summarization tasks.</li>
        <li>Inputs were tokenized with a maximum length of 2048 tokens.</li>
    Label Preparation:
        <li>Summaries were tokenized with a maximum length of 512 tokens.</li>
    Tokenization:
        <li>Used AutoTokenizer for T5-small.</li>
        <li>Applied truncation and padding to the maximum lengths.</li>
  </ul>
  
### Fine-tuning Configuration

    LoRA Configuration:
        Rank (r): 8
        Alpha: 16
        Dropout: 0.5
        Task Type: SEQ_2_SEQ_LM (Sequence-to-Sequence Language Modeling)
    Quantization Configuration:
        Load in 8-bit: True (using BitsAndBytesConfig)
    Training Arguments:
        Output Directory: ./model/t5-small-amazon-review-summarization
        Evaluation Strategy: Every 1,000 steps
        Learning Rate: 2e-7
        Per Device Training Batch Size: 2
        Per Device Evaluation Batch Size: 8
        Max Steps: 40,000
        Logging Steps: 1,000
        Save Steps: 1,000
        Load Best Model at End: True
        Predict with Generate: True
        Generation Max Length: 512
        Generation Num Beams: 5

### Training Environment

    Hardware Used: Single NVIDIA RTX 4070 8GB
    Frameworks and Libraries:
        Transformers
        Datasets
        PEFT
        BitsAndBytes
        Evaluate
        PyTorch

## Metrics

The model was tested using BERTScorer to test the accuracy of the generated summaries for the reviews.
Test set size of 3.8k pairs of reviews and summary.
  <ul>
    <li>Precision: 0.87172</li>
    <li>Recall: 0.86239</li>
    <li>F1 score: 0.86686</li>
  </ul>

## Intended Use

The model is designed to generate concise summaries of Amazon product reviews. It can be utilized in applications such as:
  <ul>
    <li>E-commerce Platforms: Summarizing customer reviews for quick insights.</li>
    <li>Product Analysis: Assisting businesses in understanding customer feedback.</li>
    <li>User Interfaces: Enhancing user experience by providing brief overviews of lengthy reviews.</li>
  </ul>

## How to Use

```python
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM, BitsAndBytesConfig
from peft import LoraConfig, PeftModel

def generate_summary(text):
    inputs = tokenizer(
        text,
        return_tensors='pt',
        padding=True,
        truncation=True,
        max_length=2048,
    ).to(device)

    summary_ids = model.generate(
        input_ids=inputs['input_ids'],
        attention_mask=inputs['attention_mask'],
        max_length=512,
        top_k=5,
        top_p=0.95,
        temperature=0.7,
        num_return_sequences=10,
        no_repeat_ngram_size=2,
        do_sample=True,
    )

    summary = [tokenizer.decode(
        summary_id,
        skip_special_tokens=True,
        clean_up_tokenization_spaces=True,
    ) for summary_id in summary_ids]
    
    return summary

tokenizer = AutoTokenizer.from_pretrained("Chryslerx10/t5-small-amazon-reviews-summarization-finetuned-8bit-lora")
model = AutoModelForSeq2SeqLM.from_pretrained(
    "t5-small",
    device_map="auto",
)

model = PeftModel.from_pretrained(model, "Chryslerx10/t5-small-amazon-reviews-summarization-finetuned-8bit-lora", device_map='auto')

generate_summary("summarize: " + ".... reviews .....")
```

## Limitations and Biases
  <ul>
    <li>Data Bias: The model was trained on a specific dataset of Amazon reviews, which may not represent all product categories or customer perspectives.</li>
    <li>Language Support: The model primarily supports English and may not perform well on reviews in other languages.</li>
    <li>Generation Quality: While the model aims to produce coherent summaries, it may occasionally generate irrelevant or nonsensical text.</li>
  </ul>