Hugging Face's logo

bigscience
/
bloomz

Text Generation
Transformers
PyTorch
TensorBoard
Safetensors
bloom
Eval Results
text-generation-inference
Model card Files Metrics Community
56
bloomz / README.md
Muennighoff's picture
Muennighoff
Create README.md
59290ca
preview code
|
raw
history blame
8.31 kB
metadata
license: bigscience-bloom-rail-1.0
datasets:
  - bigscience/xP3
language:
  - ak
  - ar
  - as
  - bm
  - bn
  - ca
  - code
  - en
  - es
  - eu
  - fon
  - fr
  - gu
  - hi
  - id
  - ig
  - ki
  - kn
  - lg
  - ln
  - ml
  - mr
  - ne
  - nso
  - ny
  - or
  - pa
  - pt
  - rn
  - rw
  - sn
  - st
  - sw
  - ta
  - te
  - tn
  - ts
  - tum
  - tw
  - ur
  - vi
  - wo
  - xh
  - yo
  - zh
  - zu
programming_language:
  - C
  - C++
  - C#
  - Go
  - Java
  - JavaScript
  - Lua
  - PHP
  - Python
  - Ruby
  - Rust
  - Scala
  - TypeScript
pipeline_tag: text-generation
widget:
  - text: >-
      A is the son's of B's uncle. What is the family relationship between A and
      B?
  - text: >-
      Reorder the words in this sentence: justin and name bieber years is my am
      I 27 old.
  - text: |-
      Task: copy but say the opposite.
       PSG won its match against Barca.
  - text: >-
      Is this review positive or negative? Review: Best cast iron skillet you
      will every buy.
    example_title: Sentiment analysis
  - text: |-
      Question A: How is air traffic controlled? 
      Question B: How do you become an air traffic controller?
      Pick one: these questions are duplicates or not duplicates.
  - text: >-
      Barack Obama nominated Hilary Clinton as his secretary of state on Monday.
      He chose her because she had foreign affairs experience as a former First
      Lady. 

      In the previous sentence, decide who 'her' is referring to.
    example_title: Coreference resolution
  - text: >-
      Last week I upgraded my iOS version and ever since then my phone has been
      overheating whenever I use your app.
       Select the category for the above sentence from: mobile, website, billing, account access.
  - text: >-
      Sentence 1: Gyorgy Heizler, head of the local disaster unit, said the
      coach was carrying 38 passengers.
       Sentence 2: The head of the local disaster unit, Gyorgy Heizler, said the bus was full except for 38 empty seats.

       Do sentences 1 and 2 have the same meaning?
    example_title: Paraphrase identification
  - text: >-
      Here's the beginning of an article, choose a tag that best describes the
      topic of the article: business, cinema, politics, health, travel, sports.

       The best and worst fo 007 as 'No time to die' marks Daniel Craig's exit.
       (CNN) Some 007 math: 60 years, 25 movies (with a small asterisk) and six James Bonds. For a Cold War creation, Ian Fleming's suave spy has certainly gotten around, but despite different guises in the tuxedo and occasional scuba gear, when it comes to Bond ratings, there really shouldn't be much argument about who wore it best.
  - text: |-
      Max: Know any good websites to buy clothes from?
       Payton: Sure :) LINK 1, LINK 2, LINK 3
       Max: That's a lot of them!
       Payton: Yeah, but they have different things so I usually buy things from 2 or 3 of them.
       Max: I'll check them out. Thanks.

       Who or what are Payton and Max referring to when they say 'them'?
  - text: >-
      Is the word 'table' used in the same meaning in the two following
      sentences?

       Sentence A: you can leave the books on the table over there.
       Sentence B: the tables in this book are very hard to read.
  - text: >-
      On a shelf, there are five books: a gray book, a red book, a purple book,
      a blue book, and a black book.
       The red book is to the right of the gray book. The black book is to the left of the blue book. The blue book is to the left of the gray book. The purple book is the second from the right.

       Which book is the leftmost book?
    example_title: Logic puzzles
  - text: >-
      The two men running to become New York City's next mayor will face off in
      their first debate Wednesday night.

       Democrat Eric Adams, the Brooklyn Borough president and a former New York City police captain, is widely expected to win the Nov. 2 election against Republican Curtis Sliwa, the founder of the 1970s-era Guardian Angels anti-crime patril.

       Who are the men running for mayor?
    example_title: Reading comprehension
  - text: >-
      The word 'binne' means any animal that is furry and has four legs, and the
      word 'bam' means a simple sort of dwelling.

       Which of the following best characterizes binne bams?
       - Sentence 1: Binne bams are for pets.
       - Sentence 2: Binne bams are typically furnished with sofas and televisions.
       - Sentence 3: Binne bams are luxurious apartments.
       - Sentence 4: Binne bams are places where people live.

Repository: bigscience-workshop/bloomz

Models

Multilingual model capable of following user instructions in a variety of languages. Together with our paper [TODO: LINK], we release the following models:

  • bloomz-p3: 176B parameter multitask finetuned version of bloom on P3. Released for research purposes, performance is inferior to bloomz
  • bloomz-7b1-p3: 7.1B parameter multitask finetuned version of bloom-7b1 on P3. Released for research purposes, performance is inferior to bloomz-7b1

Intended uses

You can use the models to perform inference on tasks by specifying your query in natural language, and the models will generate a prediction. For instance, you can ask "Translate this to Chinese: Je t'aime.", and the model will hopefully generate "我爱你".

How to use

Here is how to use the model in PyTorch:

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("bigscience/bloomz-560m")
model = AutoModelForCausalLM.from_pretrained("bigscience/bloomz-560m")

inputs = tokenizer.encode("Is this review positive or negative? Review: this is the best cast iron skillet you will ever buy", return_tensors="pt")
outputs = model.generate(inputs)
print(tokenizer.decode(outputs[0]))

To use another checkpoint, replace the path in AutoTokenizer and AutoModelForCausalLM.

Note: 176B models are trained with bfloat16, while smaller models are trained with fp16. We recommend using the same precision type or fp32 at inference

Limitations

  • Large model size may require large computational resources
  • High performance variance depending on the prompt

BibTeX entry and citation info 

TODO