File size: 3,137 Bytes
db5e7c1 9b3d759 db5e7c1 9b3d759 db5e7c1 8a38a01 9b3d759 8a38a01 9b3d759 db5e7c1 9b3d759 db5e7c1 9b3d759 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 |
---
library_name: peft
pipeline_tag: summarization
tags:
- transformers
- summarization
- dialogue-summarization
- LoRA
- PEFT
datasets:
- knkarthick/dialogsum
---
# ConvoBrief: LoRA-enhanced BART Model for Dialogue Summarization
This model is a variant of the `facebook/bart-large-cnn` model, enhanced with Low-Rank Adaptation (LoRA) for dialogue summarization tasks. LoRA employs Low-Rank Attention to facilitate feature aggregation across different positions in the sequence, making it particularly effective for capturing the nuances of dialogues.
## LoRA Configuration:
* r: 8 (Number of attention heads in LoRA)
* lora_alpha: 8 (Scaling factor for LoRA attention)
* target_modules: ["q_proj", "v_proj"] (Modules targeted for LoRA, enhancing query and value projections)
* lora_dropout: 0.05 (Dropout rate for LoRA)
* bias: "lora_only" (Bias setting for LoRA)
* task_type: Dialogue Summarization (SEQ_2_SEQ_LM)
This model has been fine-tuned using the PEFT (Parameter-Efficient Fine-Tuning) approach, striking a balance between dialogue summarization objectives for optimal performance.
## Usage:
Deploy this LoRA-enhanced BART model for dialogue summarization tasks, leveraging the power of Low-Rank Adaptation to capture contextual dependencies in conversations. Generate concise and informative summaries from conversational text, enhancing your applications with enriched context-awareness.
```python
from peft import PeftModel, PeftConfig
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
from transformers import pipeline
# Load PeftConfig and base model
config = PeftConfig.from_pretrained("Ketan3101/ConvoBrief")
base_model = AutoModelForSeq2SeqLM.from_pretrained("facebook/bart-large-cnn")
# Load PeftModel
model = PeftModel.from_pretrained(base_model, "Ketan3101/ConvoBrief")
# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained("facebook/bart-large-cnn")
# Define a pipeline for dialogue summarization
summarization_pipeline = pipeline(
"summarization",
model=model,
tokenizer=tokenizer
)
# Example dialogue for summarization
dialogue = [
#Person1#: Happy Birthday, this is for you, Brian.
#Person2#: I'm so happy you remember, please come in and enjoy the party. Everyone's here, I'm sure you have a good time.
#Person1#: Brian, may I have a pleasure to have a dance with you?
#Person2#: Ok.
#Person1#: This is really wonderful party.
#Person2#: Yes, you are always popular with everyone. and you look very pretty today.
#Person1#: Thanks, that's very kind of you to say. I hope my necklace goes with my dress, and they both make me look good I feel.
#Person2#: You look great, you are absolutely glowing.
#Person1#: Thanks, this is a fine party. We should have a drink together to celebrate your birthday
]
# Combine dialogue into a single string
full_dialogue = " ".join(dialogue)
# Generate summary
summary = summarization_pipeline(full_dialogue, max_length=150, min_length=40, do_sample=True)
print("Original Dialogue:\n", full_dialogue)
print("Generated Summary:\n", summary[0]['summary_text'])
```
### Framework versions
- PEFT 0.4.0 |