Text2Text Generation
Transformers
PyTorch
English
t5
text-generation-inference
Inference Endpoints
Edit model card

Links for Reference

TL;DR

CoT-T5 is a language model using Flan-T5 as a base model, and CoT fine-tuned on 1.84 million rationales across 1,060 tasks from the CoT Collection. Since it was CoT fine-tuned on a large amount of rationales, it shows superior performance with CoT compared to Flan-T5. One could use CoT-T5 for (1) Solving unseen tasks in zero-shot setting, and (2) Adapting to new tasks with CoT fine-tuning.

Model Details

Model Description

CoT-T5 is trained with two different sizes (3B and 11B). You could check the 3B sized LM on this page. Also, check out our dataset as well on this page.

License

CoT Collection and CoT-T5 is subject to OpenAI's Terms of Use for the generated data. If you suspect any violations, please reach out to us.

Usage

Find below some example scripts on how to use the model in transformers:

Using the Pytorch model

Running the model on a CPU

Click to expand

from transformers import T5Tokenizer, T5ForConditionalGeneration

tokenizer = T5Tokenizer.from_pretrained("kaist-ai/CoT-T5-11B")
model = T5ForConditionalGeneration.from_pretrained("kaist-ai/CoT-T5-11B")

input_text = "Read the Directions and try to pick among A,B,C,D.\n\nDirecitons: A good way to figure out the relationship in a given question is to make up a sentence that describes the relationship between the first two words. Then, try to use the same sentence to find out which of the answer choices completes the same relationship with the third word.\nQuestion: Odometer is to mileage as compass is to?\nOptions: (A) speed, (B) hiking, (C) needle, (D) direction.\nLet's think step by step.\n"
input_ids = tokenizer(input_text, return_tensors="pt").input_ids

outputs = model.generate(input_ids)
print(tokenizer.decode(outputs[0]))

Running the model on a GPU

Click to expand
# pip install accelerate
from transformers import T5Tokenizer, T5ForConditionalGeneration

tokenizer = T5Tokenizer.from_pretrained("kaist-ai/CoT-T5-11B")
model = T5ForConditionalGeneration.from_pretrained("kaist-ai/CoT-T5-11B", device_map="auto")

input_text = "Read the Directions and try to pick among A,B,C,D.\n\nDirecitons: A good way to figure out the relationship in a given question is to make up a sentence that describes the relationship between the first two words. Then, try to use the same sentence to find out which of the answer choices completes the same relationship with the third word.\nQuestion: Odometer is to mileage as compass is to?\nOptions: (A) speed, (B) hiking, (C) needle, (D) direction.\nLet's think step by step.\n"
input_ids = tokenizer(input_text, return_tensors="pt").input_ids.to("cuda")

outputs = model.generate(input_ids)
print(tokenizer.decode(outputs[0]))

Running the model on a GPU using different precisions

FP16

Click to expand
# pip install accelerate
import torch
from transformers import T5Tokenizer, T5ForConditionalGeneration

tokenizer = T5Tokenizer.from_pretrained("kaist-ai/CoT-T5-11B")
model = T5ForConditionalGeneration.from_pretrained("kaist-ai/CoT-T5-11B", device_map="auto", torch_dtype=torch.float16)

input_text = "Read the Directions and try to pick among A,B,C,D.\n\nDirecitons: A good way to figure out the relationship in a given question is to make up a sentence that describes the relationship between the first two words. Then, try to use the same sentence to find out which of the answer choices completes the same relationship with the third word.\nQuestion: Odometer is to mileage as compass is to?\nOptions: (A) speed, (B) hiking, (C) needle, (D) direction.\nLet's think step by step.\n"
input_ids = tokenizer(input_text, return_tensors="pt").input_ids.to("cuda")

outputs = model.generate(input_ids)
print(tokenizer.decode(outputs[0]))

INT8

Click to expand
# pip install bitsandbytes accelerate
from transformers import T5Tokenizer, T5ForConditionalGeneration

tokenizer = T5Tokenizer.from_pretrained("kaist-ai/CoT-T5-11B")
model = T5ForConditionalGeneration.from_pretrained("kaist-ai/CoT-T5-11B", device_map="auto", load_in_8bit=True)

input_text = "Read the Directions and try to pick among A,B,C,D.\n\nDirecitons: A good way to figure out the relationship in a given question is to make up a sentence that describes the relationship between the first two words. Then, try to use the same sentence to find out which of the answer choices completes the same relationship with the third word.\nQuestion: Odometer is to mileage as compass is to?\nOptions: (A) speed, (B) hiking, (C) needle, (D) direction.\nLet's think step by step.\n"
input_ids = tokenizer(input_text, return_tensors="pt").input_ids.to("cuda")

outputs = model.generate(input_ids)
print(tokenizer.decode(outputs[0]))

Citation

If you find the following model helpful, please considering citing our paper!

BibTeX:

@article{kim2023cot,
  title={The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuning},
  author={Kim, Seungone and Joo, Se June and Kim, Doyoung and Jang, Joel and Ye, Seonghyeon and Shin, Jamin and Seo, Minjoon},
  journal={arXiv preprint arXiv:2305.14045},
  year={2023}
}
Downloads last month
4
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for kaist-ai/CoT-T5-11B

Finetunes
1 model

Datasets used to train kaist-ai/CoT-T5-11B

Collection including kaist-ai/CoT-T5-11B