bigscience

/

bloomz

like 511

Follow

BigScience Workshop 819

Some ideas:

• outputs = model.generate(inputs, min_length=50, max_length = 1000) - force a minimum length so the model does not generate the stop token (</s>) - This should 100% work
• You can try a different prompt, e.g. Suggest at least five related search terms to "Mạng neural nhân tạo" as a comma separated list.
• You can try a larger model, which should be better

Thanks, I tried your first suggestion which produced a list.

Separately, I'd like the model to produce multiple answers to multiple questions from the same context. How best to do this without running running the same context text through the model multiple times?

Thus far, I've had success with single Q&A mimicking the P3 format:

prompt = f"""
Given the following passage 
"{context}.",
 answer the following question. Note that the answer is present within the text. 
 Question: {question}
"""

Technically, it should be feasible to reuse the cached hidden states from the context like asked here, but the generate function in transformers doesn't expose that functionality, so would require some changes to the code.

Else I'd try sth like the prompt below either zero-shot or one-shot (or more):

Zero-shot:

{context_a}

Given the above passage, please answer the below questions (the answer is in the text):
Questions:
1. XX?
2. YY?
3. ZZ?
Answers:

One-shot:

{context_a}

Given the above passage, please answer the below questions (the answer is in the text):
Questions:
1. XX?
2. YY?
3. ZZ?
Answers:
1. AA.
2. BB.
3. CC.

{context_b}

Given the above passage, please answer the below questions (the answer is in the text):
Questions:
1. WW?
2. VV?
3. UU?
Answers:
1.

Thanks for your suggestions & sorry, I missed the previous q on cached hidden states. I tried zero-shot and 1-shot(b) - they don't produce the desired output. Other suggestions?

Output:

Given the following passage 
"
I just caught this on Showtime...ewwwwwww, not even fun in a bad movie kind of way. One of the lamest monster flicks I've ever seen. Plus the TV reporter in the movie was that annoying Jerri from a past season of Survivor. The only amusing thing was that the "secret base" was the house from Fantasy Island (and a million other movies and TV shows; the place is located in the L.A. area). I fully expected Mr Roarke and Tattoo to come out and greet the visitors. If Tattoo had gotten eaten by the snake, I might have given this movie a 2, but oh well. Watching people stand there and scream for five minutes while the Komodo or the cobra loomed over them instead of making a run for it was pretty funny, especially because you could really tell that they were just screaming at an empty spot where the computer animators would later paint in the monster. I nearly fell out of my chair, though, when in a flashback scene they brought in either the cobra or the komodo - then normal size - in some indestructible solid steel container with some air holes drilled into it. Wouldn't a wire cage have sufficed? LOL! Guess they couldn't afford to rent a real komodo and cobra. I have to remember I rent Showtime for their series and not their movies.
",
answer the following questions. Note that the answers are present within the text. 
Questions:
1. What are we talking about?
2. Who are we talking about?
3. What is the sentiment expressed in this text?

Answers:
1.
positive review negative review negative review negative review negative review negative review negative review negative review negative review negative review negative review negative review negative review negative review negative review negative review negative review negative review 
<repeats...>

Hmm apart from caching which should definitely work, other ideas are:

1. Switch from greedy to sampling or increase the temparature when generating
2. Increase the model size - it should be more likely to work with the 176b model

@Muennighoff Happy holidays.
I tried to get Bloomz to generate the main points in a text with extremely limited success, hoping you have suggestions of prompts. Model: bigscience/bloomz-3b
Examples of prompts I've tried:

What are the main points in this article? Article: + text
Result: one main point (correctly from the text), then repeats that same point.

TLDR; + text
Result: one main point (correctly from the text), then "Yes. Yes. Yes"

Can you write an outline of the following article in a few points? Article: + text
Result: 3 points (all invented, not in the provided text)

@Muennighoff Happy holidays.
I tried to get Bloomz to generate the main points in a text with extremely limited success, hoping you have suggestions of prompts. Model: bigscience/bloomz-3b
Examples of prompts I've tried:

What are the main points in this article? Article: + text
Result: one main point (correctly from the text), then repeats that same point.

TLDR; + text
Result: one main point (correctly from the text), then "Yes. Yes. Yes"

Can you write an outline of the following article in a few points? Article: + text
Result: 3 points (all invented, not in the provided text)

Yeah prompt engineering can be an art & take some time. I would just try a few more prompts and else maybe increase the model size to e.g. bloomz-7b1. Some ideas could be text\n\nSummarize the above in a few bullet points. text\n\nWrite an outline of the prior article in a three bullet points.