mistralai

/

Mixtral-8x7B-Instruct-v0.1

like 4.11k

Same. Not only do I often run in to repetition until the max context length is hit, I also get these absurd hallucinations of random words:

(relatively good output)...  This should include not only a thorough examination and evaluation of the proposed approach's theoretical soundness and practical applicability but also a critical reflection on its broader ethical, social, economic, legal, environmental, cultural, political, and philosophical implications and consequences.\n\nThirdly, it is essential that the paper provides a much more explicit, detailed, comprehensive, systematic, rigorous, transparent, validated, and reproducible description and explanation of the proposed approach's theoretical foundations, assumptions, concepts, principles, methods, techniques, tools, algorithms, data structures, software architectures, system designs, implementation details, evaluation metrics, experimental results, statistical analyses, machine learning models, deep learning neural networks, natural language processing pipelines, knowledge representation and reasoning systems, information retrieval and extraction techniques, text mining and analytics methods, semantic web technologies, linked data frameworks, artificial general intelligence architectures, and other related topics.\n\nFourthly, the paper must ensure that it adequately addresses, discusses, analyzes, evaluates, critiques, questions, challenges, refutes, debunks, reconceptualizes, reinterprets, reimagines, reframes, repurposes, recontextualizes, reinserts, reconnects, recommits, restores, resuscitates, rejuvenates, rekindles, revitalizes, regenerates, reinvigorates, reenergizes, and supercharges the readers' understanding, knowledge, insights, awareness, comprehension, perception, interpretation, imagination, reasoning, judgment, decision-making, problem-solving, critical thinking, creativity, innovation, ingenuity, resourcefulness, adaptability, resilience, tenacity, determination, grit, guts, fortitude, backbone, mettle, spunk, pluck, moxie, hardiness, spirit, verve, vivacity, enthusiasm, eagerness, passion, ardor, fire, zeal, zest, avidity, hunger, thirst, craving, desire, longing, aspiration, ambition, objective, goal, end, target, aim, purpose, function, role, duty, responsibility, obligation, requirement, condition, stipulation, proviso, prerequisite, qualification, credential, license, certification, accreditation, registration, charter, authorization, endorsement, ratification, sanction, approval, agreement, consensus, unanimity, harmony, accord, peace, serenity, tranquility, calmness, stillness, quietness, repose, silence, hush, lull, whisper, murmur, rustle, crackle, buzz, hum, clink, clank, clatter, bang, knock, rap, tap, beat, drum, pulse, throb, palpitate, vibrate, oscillate, fluctuate, waver, falter, hesitate, procrastinate, dawdle, linger, loiter, tarry, saunter, stroll, swagger, amble, wander, roam, ramble, range, traverse, navigate, journey, travel, expedition, cruise, safari, pilgrimage, trek, hike, climb, scramble, mountaineer, ski, snowboard, ice skate, surf, windsurf, kitesurf, paraglide, hang glide, bungee jump, sky dive, base jump, wingsuit fly, free fall, zip line, canyon swing, giant swing, slingshot, aqua drop, human catapult, launch loop, evacuated.

Same. Not only do I often run in to repetition until the max context length is hit, I also get these absurd hallucinations of random words:

(relatively good output)...  This should include not only a thorough examination and evaluation of the proposed approach's theoretical soundness and practical applicability but also a critical reflection on its broader ethical, social, economic, legal, environmental, cultural, political, and philosophical implications and consequences.\n\nThirdly, it is essential that the paper provides a much more explicit, detailed, comprehensive, systematic, rigorous, transparent, validated, and reproducible description and explanation of the proposed approach's theoretical foundations, assumptions, concepts, principles, methods, techniques, tools, algorithms, data structures, software architectures, system designs, implementation details, evaluation metrics, experimental results, statistical analyses, machine learning models, deep learning neural networks, natural language processing pipelines, knowledge representation and reasoning systems, information retrieval and extraction techniques, text mining and analytics methods, semantic web technologies, linked data frameworks, artificial general intelligence architectures, and other related topics.\n\nFourthly, the paper must ensure that it adequately addresses, discusses, analyzes, evaluates, critiques, questions, challenges, refutes, debunks, reconceptualizes, reinterprets, reimagines, reframes, repurposes, recontextualizes, reinserts, reconnects, recommits, restores, resuscitates, rejuvenates, rekindles, revitalizes, regenerates, reinvigorates, reenergizes, and supercharges the readers' understanding, knowledge, insights, awareness, comprehension, perception, interpretation, imagination, reasoning, judgment, decision-making, problem-solving, critical thinking, creativity, innovation, ingenuity, resourcefulness, adaptability, resilience, tenacity, determination, grit, guts, fortitude, backbone, mettle, spunk, pluck, moxie, hardiness, spirit, verve, vivacity, enthusiasm, eagerness, passion, ardor, fire, zeal, zest, avidity, hunger, thirst, craving, desire, longing, aspiration, ambition, objective, goal, end, target, aim, purpose, function, role, duty, responsibility, obligation, requirement, condition, stipulation, proviso, prerequisite, qualification, credential, license, certification, accreditation, registration, charter, authorization, endorsement, ratification, sanction, approval, agreement, consensus, unanimity, harmony, accord, peace, serenity, tranquility, calmness, stillness, quietness, repose, silence, hush, lull, whisper, murmur, rustle, crackle, buzz, hum, clink, clank, clatter, bang, knock, rap, tap, beat, drum, pulse, throb, palpitate, vibrate, oscillate, fluctuate, waver, falter, hesitate, procrastinate, dawdle, linger, loiter, tarry, saunter, stroll, swagger, amble, wander, roam, ramble, range, traverse, navigate, journey, travel, expedition, cruise, safari, pilgrimage, trek, hike, climb, scramble, mountaineer, ski, snowboard, ice skate, surf, windsurf, kitesurf, paraglide, hang glide, bungee jump, sky dive, base jump, wingsuit fly, free fall, zip line, canyon swing, giant swing, slingshot, aqua drop, human catapult, launch loop, evacuated.

It seems like the context length limit is being hit every time regardless of whether it makes sense. Once the correct answer stops, it will just begin hallucinating until the limit is reached.

Because you both do not use the format you should be using. Of course it will do this when the model was trained on a certain format and you both use your custom one. You must prompt the model like this:

<s> [INST] Instruction [/INST] Model answer</s> [INST] Follow-up instruction [/INST]

Here is an example:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "mistralai/Mixtral-8x7B-Instruct-v0.1"
tokenizer = AutoTokenizer.from_pretrained(model_id)

model = AutoModelForCausalLM.from_pretrained(model_id)

text = "<s> [INST] Translate Hola to English. [/INST]"
inputs = tokenizer(text, return_tensors="pt")

outputs = model.generate(**inputs, max_new_tokens=20)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

You can also use settings like temperature, repetition_penalty top_p and top_k & others for more customized behavior.