How to use this model?

#5
by kabrav - opened
from transformers import AutoTokenizer, AutoModel

MODEL_NAME = "bigcode/starencoder"

tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)
model = AutoModel.from_pretrained(MODEL_NAME)
model.eval()

This gives

OSError: Can't load config for 'bigcode/starencoder'

Hello,

You may need to log in to hugging face's hub beforehand. This works for me: https://colab.research.google.com/drive/1lAf1c7MnzvFOTM93ciNz1HVhUvQeSy46?usp=sharing

I also added a function at the bottom showing how we prepared the tokenizer for training since it might also be useful for evaluations or fine-tuning.

Cheers

Ah and only accounts that accepted the terms of use in the model's page (https://huggingface.co/bigcode/starencoder) will be able to download and use it.

Sign up or log in to comment