Hugging Face's logo

Joemgu
/
mlong-t5-large-sumstew

Summarization
Transformers
PyTorch
Safetensors
longt5
text2text-generation
long
title generation
Eval Results
Inference Endpoints
Model card Files Community
13
mlong-t5-large-sumstew / README.md
Joemgu's picture
Joemgu
Add evaluation results on the samsum config and test split of samsum (#13)
53c5095
preview code
|
raw
history blame contribute delete
7.39 kB
metadata
language:
  - en
  - de
  - fr
  - it
  - es
license: apache-2.0
library_name: transformers
tags:
  - summarization
  - long
  - title generation
datasets:
  - Joemgu/sumstew
metrics:
  - rouge
pipeline_tag: summarization
model-index:
  - name: Joemgu/mlong-t5-large-sumstew
    results:
      - task:
          type: summarization
          name: Summarization
        dataset:
          name: samsum
          type: samsum
          config: samsum
          split: test
        metrics:
          - type: rouge
            value: 29.7108
            name: ROUGE-1
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYjQ5MjY0NzQ1NzllNGQwOThiOTM1YzEyZWVjODFjZDdjZDViMjI3ODJmMzJmMWMxMTM3MzJiMzI1MzVhYTY1NyIsInZlcnNpb24iOjF9.ba2p1M93JoZuytNmxMMEMp8XSxiTESY0fcJLLGSMinXcpNNgou5voTCBYdlSLvEKwEOHClfVdiNUVJMjzYg0BA
          - type: rouge
            value: 8.2261
            name: ROUGE-2
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZDI3MzAzNGI2MDNkY2Q5ODdiM2ZlOTM0ZDBkMDRlMjBiNzhjYmFhMmIyZTBmMjM3NjEzMDRjZTZjOTQwYTA2YiIsInZlcnNpb24iOjF9.F7ziJPm8M1RueG6vGIbaHCsq7hcp2SIi_CoQfdVSrJZbyo3wNZoWwEj3YS5AmPs7pZUYUj4d5Lyx1OzW5UInBQ
          - type: rouge
            value: 23.3337
            name: ROUGE-L
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYTA0MzRjM2JmOWJmZmM5NzhlNjIzZTBiYTg2Mjg3MzYwZTUwNDBkNjBkMDMyN2JhZjg1MzVjM2ZmNTFmM2EzOSIsInZlcnNpb24iOjF9.hwi4TH_eMUbKB-9_BxFQgpm5rTvr0-3tZXJWhJAhULXvrDaCM_QQUP15Mpvj8rhkj5RWSyyXwePXzHa2kQ5GDg
          - type: rouge
            value: 26.2366
            name: ROUGE-LSUM
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZmI1NWZhNGQwNjY1OTAxZmY5YTE0NTkyNGQ0ZTA4N2QyZTk2ZWE5NDc1ZjYwMjBmYzI1OThlN2Q4YTJhNzg0ZiIsInZlcnNpb24iOjF9.IAw2t2gIgUde3-fALzgqdxF0lj0_vkIDn350NZC7wtVa-qRBgbYc8wMOAZDz2n4B7i-vp6vbWYX19ee4gRy5Cg
          - type: loss
            value: 3.2148165702819824
            name: loss
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZGQ2ZDBhMjY4OTJmMTJiNzNkY2YxYzlmNDllY2M3YTRkYzQxOTljM2U4NzEyODUxNDMzN2E4ZWY2NjliYmQ2MCIsInZlcnNpb24iOjF9.lRdscf3-6dyITJQZc3KGIiw_hDhHSbZrN-I_8CPjeyP-x23fHSkH1UbKaYnXdzpaNwKen-FPib25rJN5mOx_CQ
          - type: gen_len
            value: 19
            name: gen_len
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMzRiYTlkOWE3YTBiMDA3MzE1MjAxYzA3N2FkNzJiNGFkYjBkNzhmY2FmMTMzYmQxNzJmMTE5ZTFmNGEwZmQ4OCIsInZlcnNpb24iOjF9.gF5iEgwXuKzCx2acampeKmEQDMd3KhH5GHHFQqVrYauTvruh43xf9fGBao-_iuSJFAnLget2kpaUISxaxUKgBA

mLong-T5-large-sumstew

TL;DR: Multilingual, longform (up to 16k input tokens), abstractive summarization model. Trained on sumstew to generate both title and summary of a given input document.

Usage:

Using pipeline (Easy): 

from transformers import pipeline

summarizer = pipeline("summarization", "joemgu/mlong-t5-large-sumstew")
text = "Alice was beginning to get very tired of sitting by her sister on the bank, and of having nothing to do: once or twice she had peeped into the book her sister was reading, but it had no pictures or conversations in it, 'and what is the use of a book,' thought Alice 'without pictures or conversations?' So she was considering in her own mind (as well as she could, for the hot day made her feel very sleepy and stupid), whether the pleasure of making a daisy-chain would be worth the trouble of getting up and picking the daisies, when suddenly a White Rabbit with pink eyes ran close by her. There was nothing so very remarkable in that; nor did Alice think it so very much out of the way to hear the Rabbit say to itself, 'Oh dear! Oh dear! I shall be late!' (when she thought it over afterwards, it occurred to her that she ought to have wondered at this, but at the time it all seemed quite natural); but when the Rabbit actually took a watch out of its waistcoat-pocket, and looked at it, and then hurried on, Alice started to her feet, for it flashed across her mind that she had never before seen a rabbit with either a waistcoat-pocket, or a watch to take out of it, and burning with curiosity, she ran across the field after it, and fortunately was just in time to see it pop down a large rabbit-hole under the hedge. In another moment down went Alice after it, never once considering how in the world she was to get out again."
summary = summarizer(text)[0]["summary_text"]
print(summary)

Output:

Title: Alice and the White Rabbit Summary: Alice is a bored and curious girl who follows a White Rabbit with a watch into a rabbit-hole. She enters a strange world where she has many adventures and meets many peculiar creatures.

Using .from_pretrained for more control (Advanced): 

from transformers import LongT5ForConditionalGeneration, T5Tokenizer

checkpoint = "joemgu/mlong-t5-large-sumstew"

gen_kwargs = {
    "max_length": 1024,
    "do_sample": False,
    "num_beams": 4, # higher = better, but uses more memory
    "use_cache": True, # Set to False if running out of memory, but will be MUCH slower
    "early_stopping": True,
    "num_return_sequences": 1,
    "repetition_penalty": 3.5,
    "encoder_repetition_penalty": 2.0,
    "length_penalty": 1.0, # higher = longer summaries
    "encoder_no_repeat_ngram_size": 4,
    "no_repeat_ngram_size": 6,
}

model = LongT5ForConditionalGeneration.from_pretrained(checkpoint)
tokenizer = T5Tokenizer.from_pretrained(checkpoint)

prefix = "Write a title and summarize: "
input_document = "Alice was beginning to get very tired of sitting by her sister on the bank, and of having nothing to do: once or twice she had peeped into the book her sister was reading, but it had no pictures or conversations in it, 'and what is the use of a book,' thought Alice 'without pictures or conversations?' So she was considering in her own mind (as well as she could, for the hot day made her feel very sleepy and stupid), whether the pleasure of making a daisy-chain would be worth the trouble of getting up and picking the daisies, when suddenly a White Rabbit with pink eyes ran close by her. There was nothing so very remarkable in that; nor did Alice think it so very much out of the way to hear the Rabbit say to itself, 'Oh dear! Oh dear! I shall be late!' (when she thought it over afterwards, it occurred to her that she ought to have wondered at this, but at the time it all seemed quite natural); but when the Rabbit actually took a watch out of its waistcoat-pocket, and looked at it, and then hurried on, Alice started to her feet, for it flashed across her mind that she had never before seen a rabbit with either a waistcoat-pocket, or a watch to take out of it, and burning with curiosity, she ran across the field after it, and fortunately was just in time to see it pop down a large rabbit-hole under the hedge. In another moment down went Alice after it, never once considering how in the world she was to get out again."

inputs = tokenizer(prefix + input_document, return_tensors="pt", max_length=16384, truncation=True, add_special_tokens=True)

outputs = model.generate(**inputs, **gen_kwargs)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Prefix your document of choice with either:

  • "Summarize: "+INPUT_TEXT
  • "Write a title and summarize: "+INPUT_TEXT

Depending on the prefix, the output will either be:

  • "Summary: SUMMARY OF INPUT_TEXT"
  • "Title: TITLE OF INPUT_TEXT Summary: SUMMARY OF INPUT_TEXT"