mistralai

/

Mistral-7B-Instruct-v0.2

like 2.57k

Follow

Mistral AI_ 2,508

I'm getting the exact same error on my side: HTTPError: 503 Server Error: Service Unavailable for url: https://api-inference.huggingface.co/models/mistralai/Mistral-7B-Instruct-v0.2

The above exception was the direct cause of the following exception:

HfHubHTTPError Traceback (most recent call last)
HfHubHTTPError: 503 Server Error: Service Unavailable for url: https://api-inference.huggingface.co/models/mistralai/Mistral-7B-Instruct-v0.2 (Request ID: 3cc2NFKMmOESTRjl-4VSi)

Model mistralai/Mistral-7B-Instruct-v0.2 is currently loading

The above exception was the direct cause of the following exception:

InferenceTimeoutError Traceback (most recent call last)
/usr/local/lib/python3.10/dist-packages/huggingface_hub/inference/_client.py in post(self, json, data, model, task, stream)
248 # If Model is unavailable, either raise a TimeoutError...
249 if timeout is not None and time.time() - t0 > timeout:
--> 250 raise InferenceTimeoutError(
251 f"Model not loaded on the server: {url}. Please retry with a higher timeout (current:"
252 f" {self.timeout}).",

InferenceTimeoutError: Model not loaded on the server: https://api-inference.huggingface.co/models/mistralai/Mistral-7B-Instruct-v0.2. Please retry with a higher timeout (current: 400).

the model didn't update since 27 day so i think the problem will solved in the close

Do you know any other free model that I can use?
I'm doing a technical project in an interview process and it's the first time I'm working with LLMs and I could make this one work from a tutorial...
I'm using this code to load it:
llm = HuggingFaceEndpoint(
timeout=400,
repo_id="mistralai/Mistral-7B-Instruct-v0.2",
max_new_tokens=240,
# top_k=10,
# top_p=0.95,
# typical_p=0.95,
temperature=0,
# repetition_penalty=1.03,
huggingfacehub_api_token='X'
)
But now I don't know any other model that should work with this code and I have until tomorrow night to finish the assignment. Can anyone help by suggesting a free alternative? Thank you!

I don't think you can use other models with the same code because the parameters of each model are different You can browse the models and read their documentation

Thanks, hopefully it is back up soon enough... I'm currently trying out the 'tiiuae/falcon-7b-instruct' but the quality of the answers is not comparable

Iguess we should just download it next time and use it locally, my whole project stopped because of this. I am using this with python.
yesterday's error
{'error': 'Model mistralai/Mistral-7B-Instruct-v0.2 is currently loading', 'estimated_time': 579.3385620117188}
today's error
{'error': 'Service Unavailable'}

they did it even without notifying lol

Yes that 's right i think we must run it localy i hope it is work soon

Okay I inspected the issue further, I guess the development team might be updating the model on hugging face right now, but It can definitely be the case that some updates are not working with hugging face at the moment, because I have found a solution to use it for now. Do you want the intermediate solution?

Yes, if you can share, that would be great!

I am using google colab because my personal computer doesn't have much power, I don't know if this complicates things.

I am using the APIs, so that I do not have to download the model myself, but planning to download as soon as it is updated on the hugging face card, and host it myself privately, to develop my app better, I do not know if the way I approach it with API would be similar to yours. Do you still want it?

the way i'm currently doing it is through the lang chain HuggingFaceEndpoint, do you think that would work? My knowledge on this framework is somewhat limited, I'm currently on the learning process so I'm sorry if I have some stupid question

I guess you are using APIs.
You used Mistral-7B-Instruct-v0.2 before langchain right?
if yes, then I can help you!
just mail me on: [email protected]
as I do not want everyone to flood that solution as well.
For me, it is working properly in my old project after adjusting.

Can you share the solution with us please

yes sure can you send me an email, or leave your email here, I really dont want this flooded, as I am really cautious if that also stops working because of the number of requests, so until hugging face api for mistral starts working again.

Ok done

Sent mail to both of you!

save me TAT hvkkvh

Just look at "files" page - https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2/tree/main
MistralAI added agreement which you need to accept. IDK if it possible via download api.

For everyone else who doesn't want to deal with this BS, here is a collection of all the models with their latest commits:
https://huggingface.co/collections/MaziyarPanahi/mistralai-6624cd82c322e4c148ca813a

They will never be removed, nor gated! Enjoy them in your production and don't let any troll tells you otherwise! You should never be grateful for unprofessional behavior, free or not!

Just look at "files" page - https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2/tree/main
MistralAI added agreement which you need to accept. IDK if it possible via download api.

Right, the people who spent time and resources to create them are simply asking for people to agree to some simple terms. And people have been adding their HF token to use API so it cant be that hard.

And those trying to circumvent, should be ashamed of yourself.

PS: People like you should contribute more, openly, freely, for years, before coming here and insult or shame others without knowing what they do!

I have, far more than you could ever comprehend. And i stopped that decades ago, to due unappreciative dickheads like you. One list time. Fuck off, self-entitled scumbag.

My actual name is none of your business. ( hint, my nick is a acronym, which clearly you dont understand. not as smart as you think you are. ha ) And for the record: No, your generation didnt 'can' me. Scumbag children like you are owned by people like me. You are just too stupid to realize it.

Next week I may want to call the HF founder ( and mistral ) about you violating terms of service around here. Ill decide later if its worth my time.

Both of you, please stop acting like kids, this a forum for help, not for petty fights.
Users who are following this thread are getting updates to your fight messages. So please refrain from doing it.
This particular thread is not because of MISTRAL's agreement and similar, it is api related, i.e. users who have already accepted the terms and conditions. So in conclusion, stop your cat fight.

Both of you, please stop acting like kids, this a forum for help, not for petty fights.
Users who are following this thread are getting updates to your fight messages. So please refrain from doing it.
This particular thread is not because of MISTRAL's agreement and similar, it is api related, i.e. users who have already accepted the terms and conditions. So in conclusion, stop your cat fight.

you are correct and apologies to everyone.

even after accepting the aggrement it says service unavailable

service unavailable

Service unavailable keeps appearing, it suggested to try a higher timeout. Does anyone know how to do that?

You can email me at: [email protected]

it fixed for me

python3 torchchat.py download mistralai/mistral-7b-instruct-v0.2
...
File "/home/dimanodg/myproject/torchchat/torchchat/cli/convert_hf_checkpoint.py", line 46, in convert_hf_checkpoint
assert len(model_map_json_matches) <= 1, "Found multiple weight mapping files"
AssertionError: Found multiple weight mapping files

Help!