
This is quantized version of the model [MBZUAI/LaMini-Flan-T5-783M](https://huggingface.co/MBZUAI/LaMini-Flan-T5-783M).

It was quantized using [CTranslate2](https://opennmt.net/CTranslate2/guides/transformers.html):

```
ct2-transformers-converter --model MBZUAI/LaMini-Flan-T5-783M --output_dir lamini-flan-t5-783m-int8_float16 --quantization int8_float16
```

# How to use it?


## Clone the model

```
git lfs install
git clone git@hf.co:saikatkumardey/lamini-flan-t5-783m_int8_float16
```

## Code example

```python
import ctranslate2
import transformers


model_dir = "lamini-flan-t5-783m_int8_float16"
translator = ctranslate2.Translator(
    model_dir, compute_type="auto", inter_threads=4, intra_threads=4
)
tokenizer = transformers.AutoTokenizer.from_pretrained("MBZUAI/LaMini-Flan-T5-783M")


input_prompt = """
instruction: Restrict the answer based on the context only. Be verbose.
context: <P>This is an introduction to pandas categorical data type, including a short comparison with R’s factor.

Categoricals are a pandas data type corresponding to categorical variables in statistics. A categorical variable takes on a limited, and usually fixed, number of possible values (categories; levels in R). Examples are gender, social class, blood type, country affiliation, observation time or rating via Likert scales.

In contrast to statistical categorical variables, categorical data might have an order (e.g. ‘strongly agree’ vs ‘agree’ or ‘first observation’ vs. ‘second observation’), but numerical operations (additions, divisions, …) are not possible.

All values of categorical data are either in categories or np.nan. Order is defined by the order of categories, not lexical order of the values. Internally, the data structure consists of a categories array and an integer array of codes which point to the real value in the categories array.

The categorical data type is useful in the following cases:

- A string variable consisting of only a few different values. Converting such a string variable to a categorical variable will save some memory, see here.
- The lexical order of a variable is not the same as the logical order (“one”, “two”, “three”). By converting to a categorical and specifying an order on the categories, sorting and min/max will use the logical order instead of the lexical order, see here.
- As a signal to other Python libraries that this column should be treated as a categorical variable (e.g. to use suitable statistical methods or plot types).</P>

question: In what cases is the categorical data type useful?
answer:
"""

input_tokens = tokenizer.convert_ids_to_tokens(tokenizer.encode(input_prompt))

results = translator.translate_batch(
    [input_tokens],
    beam_size=1,
    sampling_temperature=0.01,
    max_decoding_length=1024,
    batch_type="tokens",
    max_batch_size=64,
)

output_tokens = results[0].hypotheses[0]
output_text = tokenizer.decode(tokenizer.convert_tokens_to_ids(output_tokens))

print(output_text)
```


{"license": "cc-by-nc-4.0"}
transformers license:cc-by-nc-4.0 endpoints_compatible region:us
TAGS
#transformers #license-cc-by-nc-4.0 #endpoints_compatible #region-us 


This is quantized version of the model MBZUAI/LaMini-Flan-T5-783M.

It was quantized using CTranslate2:



# How to use it?


## Clone the model



## Code example




# How to use it? ## Clone the model ## Code example
TAGS
#transformers #license-cc-by-nc-4.0 #endpoints_compatible #region-us 
 # How to use it? ## Clone the model ## Code example