Context Limits?
#5
by
ArihantSheth
- opened
What is the context limit for this model? I am facing the following error:InternalServerError: 500 {"error":"Incomplete generation","error_type":"Incomplete generation"}
on an input of length 2727 characters containing 415 words. Model is deployed using VertexAI on Google Cloud using the following configuration:
Machine type: g2-standard-12
Accelerator type: NVIDIA_L4
Accelerator count: 1
We use 8192 as the maximum ctx length during training like Mistral and we did not revise its original config's maximum length. I think maybe that is an environment problem.
Alright, thanks
ArihantSheth
changed discussion status to
closed