randommness
i ran the model with the following parameters below 10 times in a loop and got the same exact output. dosent the model support stochastic output?
model = AutoModelForCausalLM.from_pretrained(
model_id,
trust_remote_code=True,
revision=revision,
device_map='cuda',
torch_dtype=torch.float16,
do_sample=True,
temperature=0.9,
top_p=0.9,
top_k=40
)
It needs to be passed during generation, not when initializing the model. Example:
for i in range(2):
answer = moondream.answer_question(
image_embeds, prompt, tokenizer, temperature=0.1, do_sample=True
)
print(answer)
Output:
The image features a computer server rack, which is a large metal structure designed to hold and organize multiple computer components, such as motherboards, cooling systems, and other peripherals. The rack is filled with various computer parts, including multiple computer chips, and is placed on wheels, allowing for easy movement and rearrangement of the components. The server rack is situated in a room, possibly a living room or a workspace, and is surrounded by other electronic equipment and devices.
The image features a large, black computer server rack with multiple computer boards and components on it. The rack is designed to hold and organize various electronic devices, such as computer boards, wires, and other peripherals. It is placed on a carpeted floor, indicating that it is likely in a room or workspace. The rack is filled with a variety of computer boards, suggesting that it may be used for testing, development, or maintenance purposes.
thanks its all working good now